Machine learning to predict properties of chemical formulations
Patent Information
- Application Number
- JP2023558451
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-03-25
- Filing Date
- 2021-12-15
- Publication Date
- 2024-11-28
- Estimated Expiration
- 2041-12-15
AI Technical Summary
Existing methods for predicting properties of chemical mixtures rely heavily on human data, leading to subjective biases and neglect important factors beyond perceptual similarity, such as molecular interactions and concentrations.
A computer-implemented method using machine learning, specifically graph neural networks and deep neural networks, processes molecular and mixture data to generate embeddings and predictions of properties like olfactory, catalyst, and thermal properties, considering concentration and interaction effects.
This approach provides accurate and efficient predictions of mixture properties without the need for physical testing, enabling faster and more precise formulation of chemical mixtures for various applications.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 165,781, filed March 25, 2021. U.S. Provisional Patent Application No. 63 / 165,781 is incorporated herein by reference in its entirety.
[0002] The present disclosure relates generally to predicting properties of chemical formulations using machine learning. More specifically, the present disclosure relates to property prediction using molecular, concentration, composition, and interaction properties. [Background technology]
[0003] The vast majority of chemical products are not single molecules, but carefully crafted formulations or mixtures. While the field of machine learning in chemistry has rapidly advanced to the point where it can predict the physical and sensory properties of single, isolated molecules, chemical formulations have been largely ignored.
[0004] Mixture models in the art focus on the perceptual similarity of mixtures for prediction, ignoring other factors. For example, certain existing approaches focus on storing and providing acquired human data on the properties of mixtures, such as mixtures tasted by humans. Because the stored data relies on acquired human data, subjective biases, such as different scales based on the data acquirer, may occur. Summary of the Invention [Means for solving the problem]
[0005] Aspects and advantages of embodiments of the disclosure are set forth in part in the description that follows, and may be learned from the description, or may be learned by practice of the embodiments.
[0006] One exemplary aspect of the present disclosure is directed to a computer-implemented method for mixture property prediction. The method can include obtaining, by a computing system including one or more computing devices, respective molecular data for each of a plurality of molecules and mixture data associated with a mixture of the plurality of molecules. The method can include processing, by the computing device, the respective molecular data for each of the plurality of molecules with a machine-learned embedding model to generate a respective embedding for each molecule. The method can include processing, by the computing system, the embeddings and the mixture data with a predictive model to generate one or more property predictions for the mixture of the plurality of molecules. In some implementations, the one or more property predictions can be based at least in part on the embeddings and the mixture data. The method can include storing, by the computing system, the one or more property predictions.
[0007] In some embodiments, the mixture data can describe a respective concentration of each molecule in the mixture. The mixture data can describe a composition of the mixture. The predictive model can include a deep neural network. In some embodiments, the machine-learned embedding model can include a machine-learned graph neural network. The predictive model can include a signature specific model configured to generate a prediction for a particular property. The one or more property predictions can be based at least in part on a binding energy of one or more molecules of the plurality of molecules. In some embodiments, the one or more property predictions can include one or more sensory property predictions. The one or more property predictions can include an olfactory prediction. The one or more property predictions can include a catalytic property prediction. In some embodiments, the one or more property predictions can include an energetic property prediction. The one or more property predictions can include a surfactant property prediction between targets.
[0008] In some embodiments, the one or more property predictions can include a pharmaceutical property prediction. The one or more property predictions can include a thermal property prediction. The predictive model can include a weighting model configured to weight and pool the embeddings based on mixture data, the mixture data including concentration data associated with a plurality of molecules of the mixture.
[0009] In some embodiments, the method may include obtaining, by a computing system, a request from a requesting computing device for a chemical mixture having a requested property, determining, by the computing system, whether one or more property predictions satisfy the requested property, and providing, by the computing system, the mixture data to the requesting computing device. The one or more property predictions may be based at least in part on molecular interaction properties. In some embodiments, the one or more property predictions may be based at least in part on receptor activation data.
[0010] Another exemplary aspect of the present disclosure is directed to a computing system. The computing system may include one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations may include obtaining respective molecular data for a plurality of molecules and mixture data associated with a mixture of the plurality of molecules. In some implementations, the mixture data may include a respective concentration of each molecule of the plurality of molecules. The operations may include processing the respective molecular data with an embedding model for each of the plurality of molecules to generate a respective embedding for each molecule. The operations may include processing the embedding data and the mixture data using a machine-learned predictive model to generate one or more property predictions. The one or more property predictions may be based at least in part on the embeddings and the mixture data. The operations may include storing the one or more property predictions.
[0011] Other exemplary aspects of the present disclosure are directed to one or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more processors, cause a computing system to perform operations. The operations may include obtaining respective molecular data for a plurality of molecules and mixture data associated with a mixture of the plurality of molecules. The operations may include processing the respective molecular data with an embedding model for each of the plurality of molecules to generate a respective embedding for each molecule. The operations may include processing the embedding data and the mixture data using a machine-learned predictive model to generate one or more property predictions. In some implementations, the one or more property predictions may be based at least in part on the embeddings and the mixture data. The operations may include storing the one or more property predictions.
[0012] Other aspects of the present disclosure are directed to various systems, apparatus, non-transitory computer-readable media, user interfaces, and electronic devices.
[0013] These and other features, aspects, and advantages of various embodiments of the present invention will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the present disclosure and, together with the description, serve to explain the relevant principles.
[0014] A detailed discussion of embodiments, directed to persons skilled in the art, is set forth herein with reference to the accompanying figures. [Brief description of the drawings]
[0015] [Figure 1A] FIG. 1 illustrates a block diagram of an exemplary computing system for performing mixture property prediction according to an exemplary embodiment of the present disclosure. [Figure 1B] FIG. 1 illustrates a block diagram of an exemplary computing device for performing mixture property prediction according to an exemplary embodiment of the present disclosure. [Figure 1C] FIG. 1 illustrates a block diagram of an exemplary computing device for performing mixture property prediction according to an exemplary embodiment of the present disclosure. [Diagram 2] FIG. 1 illustrates a block diagram of an exemplary machine-learned trait prediction model according to an exemplary embodiment of the present disclosure. [Diagram 3] FIG. 1 illustrates a block diagram of an exemplary trait predictive model system according to an exemplary embodiment of the present disclosure. [Figure 4] FIG. 1 illustrates a block diagram of an exemplary property request system according to an exemplary embodiment of the present disclosure. [Diagram 5] FIG. 1 illustrates a block diagram of an exemplary mixture property profile according to an exemplary embodiment of the present disclosure. [Figure 6] 1 illustrates a flowchart diagram of an exemplary method for performing mixture property prediction, according to an exemplary embodiment of the present disclosure. [Figure 7]1 illustrates a flowchart diagram of an exemplary method for performing property prediction and acquisition, according to an exemplary embodiment of the present disclosure. [Figure 8] 1 illustrates a flowchart diagram of an exemplary method for performing generation of a property prediction database, according to an exemplary embodiment of the present disclosure. [Figure 9A] FIG. 1 illustrates a block diagram of an exemplary evolutionary approach according to an exemplary embodiment of the present disclosure. [Figure 9B] FIG. 1 illustrates a block diagram of an exemplary reinforcement learning approach profile according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0016] Reference numbers repeated among the figures are intended to identify like features in various embodiments.
[0017] overview In general, the present disclosure is directed to systems and methods for predicting one or more properties of a mixture of multiple chemical molecules using machine learning. The systems and methods can utilize known properties of the individual molecules, compositions, and interactions to predict the properties of the mixture before the mixture is tested. Additionally, the machine-learned models can be used to leverage artificial intelligence techniques to quickly and efficiently predict the properties of the mixture. The systems and methods can include obtaining molecular data for one or more molecules and mixture data associated with the mixture of one or more molecules. The molecular data can include respective molecular data for each of the multiple molecules that make up the mixture. In some embodiments, the mixture data can include data related to the concentration of each molecule of the mixture along with the overall composition of the mixture. The mixture data can describe the formulation of the mixture. The molecular data can be processed with an embedding model to generate a plurality of embeddings. Each respective molecular data for each respective molecule can be processed with the embedding model to generate a respective embedding for each respective molecule in the mixture. In some embodiments, the embedding can include data describing individual molecular properties of the embedding data. In some embodiments, the embedding can be a vector of numerical values. In some cases, the embedding can represent a graph or a description of the molecular properties. The embedding data and the mixture data can be processed with a predictive model to generate one or more property predictions. The one or more property predictions can be based at least in part on the one or more embeddings and the mixture data. The property predictions can include various predictions regarding the taste, odor, color, etc. of the mixture. In some implementations, the systems and methods can include storing the one or more property predictions. In some implementations, one or both of the models can include a machine-learned model.
[0018] Obtaining the molecular data and mixture data may include receiving a request for a property prediction of a mixture including one or more molecules of the plurality of molecules. The request may further include a respective concentration of the one or more molecules. The request may include a characteristic specific property (e.g., a sensory property) or a general mixture property. Alternatively or additionally, obtaining the molecular data and mixture data may include a form of sampling, such as random sampling or category-specific sampling. For example, random sampling of the molecular mixture may be performed to catalog predictions of various mixtures. Alternatively, category-specific sampling may include taking molecules in a category of known properties and sampling with molecules in another category of other known properties.
[0019] After the molecular data is obtained, the molecular data can be processed with an embedding model to generate a plurality of embeddings. Each molecule of the plurality of molecules can receive one or more respective embeddings. The embeddings can be characteristic feature embeddings that can include embedding data related to the properties of the individual molecules. For example, the embedding of a first molecule can include embedding information that describes the olfactory properties of the molecule. In some embodiments, the embedding model can include a graph neural network that generates one or more embeddings for each molecule. In some embodiments, the embeddings can be vectors, and the vectors can be based on a processed graph, where the graph describes one or more molecules.
[0020] The one or more embeddings can be processed by a predictive model along with the mixture data to generate one or more property predictions. The predictive model can include weighting the one or more embeddings based on the concentration of the molecule with which the embedding is associated. For example, a mixture including a first molecule and a second molecule in a 2 to 1 concentration ratio can include a greater weighting for the embedding of the first molecule the higher the concentration of the first molecule in the mixture. Additionally, the machine-learned predictive model can include a weighting model that includes weighting and pooling the embeddings based on the mixture data, where the mixture data can include concentration data associated with multiple molecules of the mixture.
[0021] In some implementations, the predictive models may be machine-learned predictive models, which may include characteristic-specific models (e.g., sensory characteristic predictive models, energy characteristic predictive models, thermal characteristic predictive models, etc.).
[0022] Once generated, one or more property predictions can be stored. The predictions can be stored in a database of property predictions or can be stored on a centralized server. In some implementations, the predictions can be provided to a computing device once they are generated. The stored predictions can be organized into mixture property prediction profiles, which can include mixtures and their respective property predictions in an easy-to-understand format.
[0023] The stored predictions can be received upon request. In some embodiments, the stored predictions are easily searchable. For example, the system can receive a request for a particular property in the form of a property search query. The system can determine whether the requested property is one of the properties of the property predictions of the mixture. If the requested property is included in the property predictions, the information of the mixture can be provided to the requester.
[0024] In some embodiments, the property prediction can be based on one or more initial predictions, including, but not limited to, predicting the properties of a single molecule as a function of concentration, predicting the properties of a mixture as a function of the composition of the mixture, and predicting the properties of a mixture when the components of the mixture interact (e.g., synergistically or competitively). Each prediction may be generated by a separate model or a single model. The systems and methods may rely on fully differentiable algorithms. In some embodiments, the systems and methods may use knowledge of strong chemically induced biases and non-convex optimization to train the predictive models. Furthermore, the machine-learned models can be trained using gradient descent and a dataset of mixture data. In some embodiments, the machine-learned predictive models can be trained using a training dataset that includes labeled pairs. In some embodiments, the training data can include known receptor activation data.
[0025] In some embodiments, the systems and methods can predict the perceptual or physical properties of mixtures. The methods and systems can include explicitly modeling chemically realistic equilibrium and competitive binding dynamics, and the entire algorithm is fully differentiable. This embodiment allows for both the use of strong chemically induced biases and the use of a full toolkit of non-convex optimization techniques from the fields of neural networks and machine learning.
[0026] More specifically, machine-learned predictive models can be trained to model concentration dependence and mixtures, which may include mixtures with competitive and non-competitive inhibition. Concentration dependence can include understanding the properties of individual molecules and taking into account and weighting the properties of each molecule based on the concentration of each molecule in the mixture.
[0027] Mixtures with competitive inhibition may include mixtures in which various molecules of the mixture compete to activate receptors (e.g., molecules competing to activate odor receptors). Additionally, the system and method may take into account that molecules with higher normalized binding energy may be more likely to trigger the receptor before molecules with lower normalized binding energy. In some embodiments, mixtures with competitive inhibition may be taken into account by the system by adding a second head to the model. One head may model the net binding energy, and the other head may model the "good substrate or competitive inhibitor" propensity score, and the two heads may be multiplied element-wise. The system and method may include an attention mechanism. The two-head model may take into account which molecules activate the receptor.
[0028] Mixtures with noncompetitive inhibition may contain cumulative inhibition based on appropriate activating and noncompetitive inhibitory binding modes.
[0029] In some embodiments, the weighting of the embedding based on the concentration can be a weighted average. The weighting can produce an embedding of a single fixed dimension. In some embodiments, the concentration can be passed through nonlinearly. In some embodiments, the weighting model can produce a set of weighted graphs. Furthermore, in some embodiments, the graph structure of the molecules in the mixture can be passed to the neural network model as a weighted set, and each molecule can be organized using machine learning methods that handle variable size set inputs. For example, methods such as set2vec can be combined with graph neural network methods.
[0030] Furthermore, the graph structure of the molecules of the mixture can be embedded in a "graph of graphs", with each node representing a molecule of the mixture. Edges can be constructed in an all-to-all manner (e.g., the assumption that all types of molecules can potentially interact with each other) or can be constructed using chemical prior knowledge to filter out interactions between molecules that are more or less likely to occur. In some embodiments, edges can be weighted according to the likelihood of the interaction. Standard graph neural network methods can then be used to pass messages back and forth both within the atoms of the molecule and between the molecules as a whole.
[0031] In some implementations, the system and method may include nearest neighbor interpolation. Nearest neighbor interpolation may include enumerating a set of N components and representing each mixture as an N-dimensional vector. The vector may represent the proportion of each component. Predicting the new mixture may include a nearest neighbor search according to some distance metric, followed by averaging the perceptual characteristics of the nearest neighbors. The averaged perceptual characteristics may be the prediction.
[0032] Alternatively or additionally, in some embodiments, the systems and methods can include direct molecular dynamics simulations using quantum mechanics-based or molecular force field-based approaches. For example, the interaction of each molecule with a putative olfactory or gustatory receptor can be directly modeled using a dedicated computer for molecular simulation, and the strength of the interaction can be measured by simulation. The sensory properties of a mixture can be modeled based on the combined interactions of all components.
[0033] The property predictions may include sensory property predictions (e.g., olfactory properties, taste properties, color properties, etc.). Additionally and / or alternatively, the property predictions may include catalytic property predictions, energetic property predictions, target-target surfactant property predictions, pharmaceutical property predictions, odor properties predictions, odor intensity predictions, color predictions, viscosity predictions, lubricant property predictions, boiling point predictions, adhesion property predictions, colorability predictions, stability predictions, and thermal property predictions. For example, the property predictions may include predictions related to properties that may be beneficial in the design of a battery, such as the time the mixture will hold a charge, the amount of charge the mixture can hold, discharge, rate, degradation rate, stability, overall quality, etc.
[0034] The systems and methods disclosed herein can be applied to generate property predictions for a variety of applications, including, but not limited to, industrial applications such as consumer packaging, flavors and fragrances, dyes, paints, lubricants, and energy applications such as battery design.
[0035] In some embodiments, the systems and methods described herein may be implemented by one or more computing devices. The computing device (a) may include one or more processors and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the computing device to perform operations. The operations may include steps of various methods described herein.
[0036] In some embodiments, the systems and methods disclosed herein can be used in a closed-loop development process. For example, a human practitioner can utilize the systems and methods disclosed herein to predict the properties of a mixture before physically creating the mixture. In some embodiments, the systems and methods can be used to generate a database of theoretical mixtures with predicted properties. A human practitioner can utilize the generated database to enable computer-aided design of mixtures to achieve a desired effect. Additionally, the database can be a searchable database that can be used to screen all possible mixtures to identify mixtures with desired sensory and physical properties.
[0037] For example, a human practitioner may be attempting to create a new, powerful, flowery fragrance. The human practitioner may provide theoretical mixture suggestions to the embedding model and the machine learning predictive model to output predicted properties of the theoretical mixture. The human practitioner may use the prediction to determine whether to actually manufacture the mixture or to continue formulating other mixtures for testing. In some embodiments, in response to determining that one or more mixtures are predicted to have desired properties, the system may send instructions to a manufacturing system or a user computing system to manufacture one or more mixtures for physical testing.
[0038] Alternatively and / or additionally, a human practitioner can search or screen mixtures that have already been processed by a machine learning model(s) to generate property predictions. The mixtures and their respective property predictions may be stored in a database to facilitate data screening or searching. A human practitioner can screen multiple mixtures to find a mixture whose property predictions match the desired properties. For example, a human practitioner seeking to create a new strong floral fragrance may screen a database for mixtures that are predicted to have a strong odor with floral characteristics.
[0039] Utilizing the closed-loop development process of the systems and methods disclosed herein can save time and costs of manufacturing and physically testing mixtures. A human practitioner can use the machine-learned model to screen the data and quickly eliminate a large number of possible mixtures from the pool of possible candidates. Additionally, the machine-learned model may predict properties indicative of candidate mixtures that a human practitioner may overlook because the candidate mixtures have unexpected cumulative properties.
[0040] In some embodiments, the system and method for predicting one or more properties of a mixture of multiple chemical molecules using machine learning may be used to control machinery and / or provide an alert. The system and method may be used to control manufacturing machinery to provide a safer working environment or to modify the composition of the mixture to provide a desired production yield. Additionally, in some embodiments, the property prediction may be processed to determine whether an alert needs to be provided. For example, in some embodiments, the property prediction may include an olfactory property prediction for the scent of a vehicle used for transportation services. The system and method may output a scent profile prediction, a potency prediction, and a scent life prediction for an air freshener, fragrance, or candle replacement. The prediction may then be processed to determine when a new product should be placed on the transportation device and / or whether the transportation device should undergo a cleaning routine. The determined new product time may then be sent as an alert to a user's computing device or may be used to set up an automatic purchase. In another example, the transportation device (e.g., an autonomous vehicle) may be automatically recalled to a facility to undergo a cleaning routine. In another example, a property prediction generated by a machine learning model may provide an alert indicating a dangerous environment for animals or people present in the space. For example, an audio alarm may be sounded in a building if a prediction is generated that the building is unsafe for a mixture of chemical molecules sensed to be present in the building.
[0041] In some implementations, the system can incorporate sensor data that is input into the embedded model and the predictive model to generate a property prediction for the environment. For example, the system can utilize one or more sensors to incorporate data related to the presence and / or concentration of molecules in the environment. The system can process the sensor data to generate input data for the embedded model and process the predictive model to generate a property prediction for the environment. This can include one or more predictions regarding the odor or other properties of the environment. If the prediction includes a particular unpleasant odor, the system can send an alert to the user's computing device to complete the cleaning service. In some implementations, upon determining an unpleasant odor, the system can bypass the alert and send a booking request to the cleaning service.
[0042] Another exemplary embodiment may include background processing and / or active monitoring for safety measures. For example, the system may document manufacturing steps completed by a user or machine and track predicted properties of mixtures created to ensure the manufacturer is aware of any hazards. In some embodiments, upon selection of new molecules or mixtures to be added to an ongoing mixture, new potential mixtures may be processed by the embedded and predictive models to determine property predictions for the new mixture. Property predictions may include whether the new mixture is flammable, toxic, unstable, or in any way hazardous. If the new mixture is determined to be hazardous in any way, an alert may be sent. Alternatively and / or additionally, the system may control one or more machines to stop and / or contain the process to protect against any potential current or future hazards.
[0043] The systems and methods can be applied to other manufacturing, industrial, or commercial systems to provide automated alerts or automated actions in response to property predictions. These applications can include creating new mixtures, recipe adjustments, countermeasures, and real-time alerts for changes in predicted properties.
[0044] The disclosed system and method provide many technical effects and advantages. As an example, the system and method can provide a prediction of the properties of a mixture without the need to individually and physically test various mixtures of molecules. The system and method can further be used to generate a database of mixtures with predicted properties, which can be easily searchable to find mixtures with specific properties to be introduced into fragrances, foods, lubricants, etc. based on the predicted properties. Furthermore, the system and method can enable more accurate predictions by considering both individual molecular properties and interaction properties. Thus, the ability of a computer to perform a task (e.g., predicting the scent of a mixture) can be improved.
[0045] Another technical advantage of the systems and methods of the present disclosure is that they can quickly and efficiently predict the properties of mixtures, thereby avoiding the need to test the mixtures in human taste tests and other physical testing applications.
[0046] Referring now to the drawings, exemplary embodiments of the present disclosure are described in further detail.
[0047] Device and System Examples 1A illustrates a block diagram of an example computing system 100 for performing trait prediction in accordance with an example embodiment of the present disclosure. The system 100 includes a user computing device 102, a server computing system 130, and a training computing system 150 communicatively coupled via a network 180.
[0048] The user computing device 102 may be any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop), a mobile computing device (e.g., a smartphone or tablet), a game console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.
[0049] The user computing device 102 includes one or more processors 112 and memory 114. The one or more processors 112 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be a single processor or multiple processors operatively connected. The memory 114 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 114 may store data 116 and instructions 118 that are executed by the processor 112 to cause the user computing device 102 to perform operations.
[0050] In some implementations, the user computing device 102 may store or include one or more predictive models 120. For example, the predictive models 120 may be or otherwise include various machine-learned models, such as neural networks (e.g., deep neural networks), or other types of machine-learned models including non-linear and / or linear models. The neural networks may include feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, or other types of neural networks. Exemplary predictive models 120 are described with reference to FIGS. 2, 3, and 6-8.
[0051] In some implementations, one or more predictive models 120 may be received from the server computing system 130 over the network 180 and stored in the user computing device memory 114 and then used or otherwise implemented by one or more processors 112. In some implementations, the user computing device 102 may implement multiple parallel instances of a single predictive model 120 (e.g., to perform parallel mixture property predictions across multiple instances of the mixture's composition).
[0052] More specifically, a machine-learned predictive model can be trained to take in molecular data and mixture data and output property predictions for the mixtures described by the mixture data. In some implementations, the molecular data may be embedded with an embedding model before being processed by the predictive model.
[0053] Additionally or alternatively, one or more predictive models 140 may be included in or otherwise stored and implemented on a server computing system 130 that communicates with the user computing device 102 according to a client-server relationship. For example, the predictive models 140 may be implemented by the server computing system 140 as part of a web service (e.g., a mixture property prediction service). Thus, one or more models 120 may be stored and implemented on the user computing device 102 and / or one or more models 140 may be stored and implemented on the server computing system 130.
[0054] The user computing device 102 may also include one or more user input components 122 that receive user input. For example, the user input component 122 may be a touch-sensitive component (e.g., a touch-sensitive display screen or touchpad) that senses the touch of a user input object (e.g., a finger or stylus). The touch sensor component may function to implement a virtual keyboard. Other exemplary user input components include a microphone, a traditional keyboard, or other means by which a user can provide user input.
[0055] The server computing system 130 includes one or more processors 132 and memory 134. The one or more processors 132 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be a single processor or multiple processors operatively connected. The memory 134 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 134 may store data 136 and instructions 138 that are executed by the processor 132 to cause the server computing system 130 to perform operations.
[0056] In some implementations, server computing system 130 includes or is implemented by one or more server computing devices. When server computing system 130 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or a combination thereof.
[0057] As described above, the server computing system 130 may store or otherwise include one or more machine-learned predictive models 140. For example, the models 140 may be or include a variety of machine-learned models. Exemplary machine-learned models include neural networks and other multi-layer nonlinear models. Exemplary neural networks include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Exemplary models 140 are described with reference to FIGS. 2, 3, and 6-8.
[0058] The user computing device 102 and / or the server computing system 130 can train the models 120 and / or 140 through interaction with a training computing system 150 that is communicatively coupled via a network 180. The training computing system 150 may be separate from the server computing system 130 or may be part of the server computing system 130.
[0059] The training computing system 150 includes one or more processors 152 and memory 154. The one or more processors 152 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be a single processor or multiple processors operatively connected. The memory 154 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 154 may store data 156 and instructions 158 that are executed by the processor 152 to cause the training computing system 150 to perform operations. In some implementations, the training computing system 150 includes or is implemented by one or more server computing devices.
[0060] The training computing system 150 may include a model trainer 160 that trains the machine-learned models 120 and / or 140 stored on the user computing device 102 and / or the server computing system 130 using various training or learning techniques, such as, for example, backpropagation of errors. For example, a loss function may be backpropagated through the model(s) to update one or more parameters of the model(s) (e.g., based on the gradient of the loss function). Various loss functions may be used, such as mean squared error, likelihood loss, cross-entropy loss, hinge loss, and / or various other loss functions. Using gradient descent, the parameters may be iteratively updated over multiple iterations of training.
[0061] In some implementations, performing backpropagation of the error may include performing time-saving backpropagation. The model trainer 160 may implement a number of generalization techniques (e.g., weight decay, dropout, etc.) to improve the generalization ability of the model being trained.
[0062] In particular, model trainer 160 can train predictive models 120 and / or 140 based on a set of training data 162. Training data 162 can include labeled training data, such as, for example, molecular data with known molecular feature labels, mixture data with known composition feature labels, and mixture data with known interaction feature labels.
[0063] In some implementations, if the user consents, the training examples may be provided by the user computing device 102. Thus, in such implementations, the model 120 provided to the user computing device 102 may be trained by the training computing system 150 on user-specific data received from the user computing device 102. This process is sometimes referred to as personalizing the model.
[0064] Model trainer 160 includes computer logic utilized to effect desired functionality. Model trainer 160 may be implemented in hardware, firmware, and / or software controlling a general-purpose processor. For example, in some implementations, model trainer 160 includes program files stored in a storage device, loaded into memory, and executed by one or more processors. In other implementations, model trainer 160 includes one or more sets of computer-executable instructions stored in a tangible computer-readable storage medium, such as RAM, a hard disk, an optical medium, or a magnetic medium.
[0065] Network 180 may be any type of communications network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and may include any number of wired or wireless links. In general, communications over network 180 may be performed over any type of wired and / or wireless connections, using a wide variety of communications protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, Secure HTTP, SSL, etc.).
[0066] The machine-learned models described herein can be used in a variety of tasks, applications, and / or use cases.
[0067] In some implementations, the input to the machine-learned model(s) of the present disclosure may be image data. The machine-learned model(s) may process the image data to generate an output. As an example, the machine-learned model(s) may process the image data to generate an image recognition output (e.g., a recognition of the image data, a latent embedding of the image data, an encoded representation of the image data, a hash of the image data, etc.). As another example, the machine-learned model(s) may process the image data to generate a molecular graph output, which may then be processed by an embedding model and a predictive model to generate a property prediction.
[0068] In some implementations, the input to the machine-learned model(s) of the present disclosure may be text or natural language data. The machine-learned model(s) may process the text or natural language data to generate an output. As an example, the machine-learned model(s) may process the natural language data to generate a search query output. The search query output may be processed by the search model to search for mixtures with a particular property and output one or more mixtures with the particular property. In another example, the machine-learned model(s) may process the text or natural language data to generate a classification output. The classification output may describe the mixtures with one or more predicted properties. In another example, the machine-learned model(s) may process the text or natural language data to generate a prediction output.
[0069] In some implementations, the input to the machine-learned model(s) of the present disclosure may be latent encoded data (e.g., a latent space representation of the input, etc.). The machine-learned model(s) may process the latent encoded data to generate an output. As an example, the machine-learned model(s) may process the latent encoded data to generate a recognition output. As another example, the machine-learned model(s) may process the latent encoded data to generate a reconstruction output. As another example, the machine-learned model(s) may process the latent encoded data to generate a search output. As another example, the machine-learned model(s) may process the latent encoded data to generate a reclustering output. As another example, the machine-learned model(s) may process the latent encoded data to generate a prediction output.
[0070] In some implementations, the input to the machine-learned model(s) of the present disclosure may be statistical data. The machine-learned model(s) may process the statistical data to generate an output. As an example, the machine-learned model(s) may process the statistical data to generate a recognition output. As another example, the machine-learned model(s) may process the statistical data to generate a prediction output. As another example, the machine-learned model(s) may process the statistical data to generate a classification output. As another example, the machine-learned model(s) may process the statistical data to generate a segmentation output. As another example, the machine-learned model(s) may process the statistical data to generate a segmentation output. As another example, the machine-learned model(s) may process the statistical data to generate a visualization output. As another example, the machine-learned model(s) may process the statistical data to generate a diagnostic output.
[0071] In some implementations, the input to the machine-learned model(s) of the present disclosure may be sensor data. The machine-learned model(s) may process the sensor data to generate an output. As an example, the machine-learned model(s) may process the sensor data to generate a recognition output. As another example, the machine-learned model(s) may process the sensor data to generate a prediction output. As another example, the machine-learned model(s) may process the sensor data to generate a classification output. As another example, the machine-learned model(s) may process the sensor data to generate a segmented output. As another example, the machine-learned model(s) may process the sensor data to generate a segmented output. As another example, the machine-learned model(s) may process the sensor data to generate a visualization output. As another example, the machine-learned model(s) may process the sensor data to generate a diagnostic output.
[0072] In some cases, the input includes visual data and the task is a computer vision task. In some cases, the input includes pixel data of one or more images and the task is an image processing task. For example, the image processing task can be image classification and the output is a set of scores, each score corresponding to a different object class and representing the likelihood that one or more images depict an object belonging to that object class. The image processing task can be object detection and the image processing output identifies one or more regions of one or more images and, for each region, the likelihood that the region represents an object of interest. As another example, the image processing task can be image segmentation and the image processing output defines, for each pixel of one or more images, a respective likelihood for each category of a set of predefined categories. As another example, the set of categories can be object classes.
[0073] 1A illustrates an example of a computing system that can be used to implement the present disclosure. Other computing systems can be used as well. For example, in some implementations, the user computing device 102 can include a model trainer 160 and a training dataset 162. In such implementations, the model 120 can be trained and used locally on the user computing device 102. In some such implementations, the user computing device 102 can implement the model trainer 160 to personalize the model 120 based on user-specific data.
[0074] 1B illustrates a block diagram of an exemplary computing device 10 for performing according to an exemplary embodiment of the present disclosure. The computing device 10 may be a user computing device or a server computing device.
[0075] The computing device 10 includes multiple applications (e.g., applications 1 through N). Each application includes its own machine learning library and machine-learned model(s). For example, each application may include a machine-learned model. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc.
[0076] 1B, each application may communicate with many other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, each application may communicate with each device component using an API (e.g., a public API). In some implementations, the API used by each application is specific to that application.
[0077] 1C illustrates a block diagram of an exemplary computing device 50 for performing according to an exemplary embodiment of the present disclosure. The computing device 50 may be a user computing device or a server computing device.
[0078] Computing device 50 includes multiple applications (e.g., applications 1 through N). Each application communicates with a central intelligence layer. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc. In some implementations, each application can communicate with the central intelligence layer (and the model(s) stored therein) using an API (e.g., a common API across all applications).
[0079] The central intelligence layer includes multiple machine-learned models. For example, as shown in FIG. 1C, each machine-learned model (e.g., model) can be provided for each application and managed by the central intelligence layer. In other embodiments, two or more applications can share a single machine-learned model. For example, in some embodiments, the central intelligence layer can provide a single model (e.g., a single model) for all applications. In some embodiments, the central intelligence layer is included within or otherwise implemented by an operating system of the computing device 50.
[0080] The central intelligence layer can communicate with a central device data layer, which can be a central repository of data for computing device 50. As shown in FIG. 1C, the central device data layer can communicate with many other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, the central device data layer can communicate with each device component using an API (e.g., a private API).
[0081] Example model layout In some embodiments, the system and method may include a graph neural network (GNN) and a deep neural network (DNN) to process data. The system and method may take into account the normalized binding energy (NBE) and the concentration of molecules in a mixture to better understand the mixture and how the mixture may behave. The graph neural network (GNN), the deep neural network (DNN), and the normalized binding energy (NBE) may be written as their respective acronyms, and the concentration may be written as [X] for the concentration of X.
[0082] In some embodiments, the system may include considering concentration dependency in the prediction and then modeling the mixture as a whole. The system may include processing the molecular data with a GNN to generate molecular embeddings (i.e., molecule_embedding=GNN(molecule)). The molecular embeddings may then be processed with a DNN to generate NBE data (i.e., NBE=DNN(molecule_embedding)). The NBE of the molecule and the concentration of the molecule in the mixture may then be processed by various layers, which may include a softmax layer, and pooled with all other processed NBEs and concentrations of other molecules in the mixture to generate receptor activation data (e.g., receptor_activations=sum(softmax([NBE+log[M],0])[:-1])). In some embodiments, the generated receptor activation data may then be processed with a DNN to generate perceptual odor response data (i.e., perceptual_odor_response=DNN(receptor_activations)). Alternatively and / or additionally, the system may simplify the process to include processing the molecular data with a GNN to generate molecular embeddings (i.e., molecule_embedding = GNN(molecule)), and then processing the molecular embeddings with a DNN to generate perceptual odor response data (i.e., perceptual_odor_response = DNN(molecule_embedding)).
[0083] In some embodiments, the systems and methods can determine proper substrate scores and / or generate feature vectors to aid in mixture modeling and generating property predictions. In some embodiments, proper substrate scores can be determined by processing molecule embeddings with a DNN, applying a sigmoid activation function, and concatenating the results (e.g., proper_substrate_score=concat(sigmoid(DNN(molecule_embedding)),[0])). Similarly, feature vectors can be generated using the concentrations of molecules, the normalized binding energies of molecules, and a softmax activation function (e.g., OR_vector=softmax([NBE+log[M],0])). For mixture modeling, the proper substrate scores and feature vectors can then be used to determine receptor activation data by scaling the vectors by the scores and then summing the results (e.g., receptor_activations=sum(proper_substrate_score*OR_vector)). Furthermore, the receptor activation data can then be used to determine the perceptual odor response data (e.g., perceptual_odor_response=DNN(receptor_activations)).
[0084] In some implementations, the inhibition of molecules can be factored into the prediction. For example, the system and method can determine inhibition data related to normalized binding energies through a process similar to determining the normalized binding energies of molecules. The molecular data can be processed with a GNN to generate molecular embeddings, and then the molecular embeddings can be processed with a DNN to generate inhibition data. This data can be represented as inhibition_NBE=DNN(molecule_embedding). The inhibition data can then be used to determine receptor inhibition data by processing the inhibition data and concentration data for each molecule through various layers, including a softmax layer, and summing the results (e.g., receptor_inhibitions=sum(softmax([inhibition_NBE+log[M],0])[:-1])). The receptor activation data and receptor inhibition data can be used to calculate net receptor activation data (e.g., net_receptor_activations=receptor_activations*(1-receptor_inhibitions)). This can be used to generate perceptual odor response data with a DNN (e.g., perceptual_odor_response = DNN(net_receptor_activations)).
[0085] In some embodiments, each sensory odor response function and model may be incorporated into an overall property prediction for a mixture, for example, concentration dependence, mixtures with competitive inhibition, and mixtures with non-competitive inhibition may be incorporated into an overall machine-learned predictive model using various functions, architectures, and models.
[0086] In some implementations, the systems and methods may include a specialized framework for processing molecules individually to determine individual properties of the molecules using an embedding model or a first machine-learned model. These systems and methods may include or otherwise utilize a machine-learned model (e.g., a graph neural network) in conjunction with the chemical structure data of the molecule to predict one or more sensory (e.g., olfactory, gustatory, tactile, etc.) properties of the molecule. In particular, the systems and methods may predict the olfactory properties of a single molecule (e.g., the odor as perceived by humans, described using labels such as "sweet," "pine-like," "pear," "rotten," etc.) based on the chemical structure of the molecule. Furthermore, in some implementations, a machine-learned graph neural network may be trained and used to process a graph that graphically describes the chemical structure of a molecule to predict the olfactory properties of the molecule. In particular, the graph neural network may operate directly on a graph representation of the chemical structure of a molecule (e.g., perform a convolution in the graph space) to predict the olfactory properties of the molecule. As an example, the graph may include nodes corresponding to atoms and edges corresponding to chemical bonds between the atoms. Thus, the systems and methods of the present disclosure can provide predictive data to predict the odor of previously unassessed molecules through the use of machine-learned models. The machine-learned models for individual molecules can be trained, for example, using training data that includes descriptions of molecules (e.g., structural descriptions of molecules, graph-based descriptions of the chemical structure of molecules, etc.) that have been labeled (e.g., manually by an expert) with descriptions of the olfactory properties assessed for the molecules (e.g., textual descriptions of odor categories such as "sweet," "pine-like," "pear," "rotten," etc.).
[0087] Thus, the first machine-learned model, the embedding model, may use a graph neural network for quantitative structure-odor relationship (QSOR) modeling. The embeddings learned from the graph neural network capture a meaningful odor space representation that represents the underlying relationships between structures and odors.
[0088] More specifically, the relationship between the structure of a molecule and its olfactory properties (e.g., the scent of the molecule as observed by humans) is complex, and to date, little is generally known about such relationships. Thus, the systems and methods of the present disclosure provide for the use of deep learning and underutilized data sources to obtain predictions of the olfactory sensory properties of unseen molecules, thereby enabling improved identification and development of molecules with desired sensory properties. For example, this allows for the development of new compounds useful in commercial flavors, fragrances, or cosmetics, improved expertise in predicting the psychoactive effects of drugs from single molecules, etc.
[0089] More specifically, according to one aspect of the present disclosure, a machine-learned model, such as a graph neural network model, can be trained to provide predictions of sensory properties (e.g., olfactory properties, gustatory properties, tactile properties, etc.) of a molecule based on an input graph of the molecule's chemical structure. For example, the machine-learned model can be provided with an input graph structure of the molecule's chemical structure, for example, based on a standardized description of the molecule's chemical structure (e.g., a Simple Molecular Input Line Entry System (SMILES) string, etc.). The machine-learned model can provide an output including a description of the predicted sensory properties of the molecule, for example, a list of olfactory properties that describe how the molecule smells to a human. For example, a SMILES string, such as the SMILES string "O=C(OCCC(C)C)C" representing the chemical structure of isoamyl acetate, can be provided, and the machine-learned model can provide as an output a description of how the molecule would smell to a human, for example, a description of the molecular odor properties, such as "fruit, banana, apple." In particular, in some embodiments, in response to receiving a SMILES string or other description of a chemical structure, the systems and methods can convert the string into a graph structure that graphically describes the two-dimensional structure of the molecule, and can provide the graph structure with a machine-learned model (e.g., a trained graph convolutional neural network and / or other type of machine-learned model) that can predict the olfactory properties of the molecule from either the graph structure or features derived from the graph structure. In addition to or instead of a two-dimensional graph, the systems and methods can provide for creating a three-dimensional graph representation of the molecule as input to the machine-learned model, e.g., using quantum chemistry calculations.
[0090] In some examples, the prediction may indicate whether a molecule has a particular desired olfactory sensory property (e.g., the perception of a target scent, etc.). In some embodiments, the prediction data may include one or more types of information related to the predicted olfactory sensory property of the molecule. For example, the prediction data of a molecule may result in a classification of the molecule into one olfactory sensory property class and / or multiple olfactory sensory property classes. In some cases, the classes may include human (e.g., expert) provided text labels (e.g., sour, cherry, piney, etc.). In some cases, the classes may include non-textual representations of the scent / odor, such as a position in a scent continuum. In some cases, the prediction data of a molecule may include an intensity value representing the intensity of the predicted scent / odor. In some cases, the prediction data may include a confidence value associated with the predicted olfactory sensory property.
[0091] In addition to or instead of a particular classification of molecules, the prediction data can include a numerical embedding that allows for similarity searching, clustering, or other comparison between two or more molecules based on a distance measure between the two or more embeddings. For example, in some embodiments, the machine-learned model can be trained to output embeddings that can be used to measure similarity by training the machine-learned model using a triplet training scheme. In this scheme, the model is trained to output embeddings that are closer in the embedding space for pairs of similar chemical structures (e.g., anchor examples and positive examples) and to output embeddings that are farther away in the embedding space for pairs of dissimilar chemical structures (e.g., anchor and negative examples). Furthermore, the output of these models can be configured to be processed by a second machine-learned model to predict the properties of a mixture of various models.
[0092] Therefore, in some embodiments, the system and method of the present disclosure may not require the generation of a feature vector describing a molecule for input to a machine-learned model.Rather, the machine-learned model can be directly provided with input in the form of graph values of original chemical structures, thereby reducing the resources required for predicting olfactory properties.For example, by allowing the graph structure of molecules to be used as input to a machine-learned model, new molecular structures can be conceptualized and evaluated without the need to experimentally create such molecular structures to determine sensory properties, greatly improving the ability to evaluate new molecular structures and greatly saving resources.
[0093] Further, in some embodiments, training data including a plurality of known molecules can be obtained to train one or more machine-learned models (e.g., graph convolutional neural networks, other types of machine-learned models) to provide predictions of the olfactory properties of the molecules. For example, in some embodiments, the machine-learned models can be trained using one or more datasets of molecules, where the datasets can include, for each molecule, a chemical structure and a textual description of the sensory properties (e.g., a description of the odor of the molecule provided by a human expert, etc.). As an example, the training data can be obtained from an industry list, such as, for example, a publicly available perfume industry list of chemical structures and their corresponding odors. In some embodiments, due to the fact that some sensory properties are rare, steps can be taken to balance common sensory properties with rare sensory properties when training the machine-learned model(s).
[0094] According to another aspect of the present disclosure, in some embodiments, the system and method can provide an indication of how changes in molecular structure affect predicted sensory properties. These changes can then be processed by a second machine-learned model to generate interaction property predictions, which can be used to generate an overall mixture property prediction. For example, the system and method can provide an indication of how changes in molecular structure can affect the intensity of a particular sensory property, how devastating the impact of changes in molecular structure is on the desired sensory property, etc. In some implementations, the system and method can provide for adding and / or removing one or more atoms and / or groups of atoms from the structure of the molecule to determine the effect of such addition / removal on one or more desired sensory properties. For example, various modifications to the chemical structure can be iteratively performed and then the results evaluated to understand how such modifications affect the sensory properties of the molecule. As yet another example, the gradient of the classification function of the machine-learned model can be evaluated (e.g., with respect to a particular label) at each node and / or edge of the input graph (e.g., via backpropagation through the machine-learned model) to generate a sensitivity map (e.g., indicating how important each node and / or edge of the input graph is to the output of such particular label). Further, in some implementations, a graph of a subject can be taken, similar graphs can be sampled by adding noise to the graph, and then an average of the resulting sensitivity maps for each sampled graph can be taken as the sensitivity map of the subject graph. Similar techniques can be performed to determine perceptual differences between different molecular structures.
[0095] In some embodiments, the system and method can provide an interpretation and / or visualization of which aspects of a molecule's structure contribute most to the predicted odor quality. For example, in some embodiments, a heat map can be generated to overlay a molecular structure that provides an indication of which parts of the molecule's structure are most important to the sensory properties of the molecule and / or which parts of the molecule's structure are less important to the sensory properties of the molecule. In some embodiments, data showing how changes in a molecular structure affect olfaction can be used to generate a visualization of how the structure contributes to the predicted olfactory quality. For example, as described above, iterative changes to the structure of a molecule (e.g., knockdown techniques, etc.) and their corresponding results can be used to evaluate which parts of a chemical structure contribute most to olfaction. As another example, as described above, a gradient technique can be used to generate a sensitivity map of a chemical structure, which can then be used to generate a visualization (e.g., in the form of a heat map).
[0096] Further, in some implementations, the machine-learned model(s) may be trained to generate predictions of chemical structures of molecules that provide one or more desired sensory properties (e.g., generate chemical structures of molecules that produce a particular scent quality, etc.). For example, in some implementations, an iterative search may be performed to identify proposed molecule(s) that are predicted to exhibit one or more desired sensory properties (e.g., a target scent quality, intensity, etc.). For example, the iterative search may propose a number of candidate molecular chemical structures that can be evaluated by the machine-learned model(s). In one example, the candidate molecular structures may be generated through an evolutionary or genetic process. As another example, the candidate molecular structures may be generated by a reinforcement learning agent (e.g., a recurrent neural network) that attempts to learn a policy that maximizes a reward that is a function of whether the generated candidate molecular structures exhibit one or more desired sensory properties.
[0097] Thus, in some embodiments, multiple candidate molecule graph structures describing the chemical structure of each candidate molecule can be generated (e.g., iteratively generated) for use as input to the machine-learned model. The graph structures of each candidate molecule can be input to the machine-learned model for evaluation. The machine-learned model can generate prediction data for a group of molecules describing one or more sensory properties of each candidate molecule or one or more candidate molecules. The candidate molecule prediction data can then be compared to one or more desired sensory properties to determine whether the candidate molecule(s) exhibit the desired sensory properties (e.g., viable molecular candidates, etc.). For example, the comparison can be performed to generate rewards (e.g., in a reinforcement learning scheme) or to determine whether to keep or discard the candidate molecule (e.g., in an evolutionary learning scheme). Brute force search approaches can also be used. In further embodiments, with or without the above evolutionary or reinforcement learning structures, the search for candidate molecules exhibiting one or more desired sensory properties can be structured as a multi-parameter optimization problem with constraints for optimization defined for each desired property.
[0098] The systems and methods may enable prediction, identification, and / or optimization of other properties associated with molecular structures along with desired olfactory properties. For example, the machine learning model(s) may predict or identify properties of molecular structures such as optical properties (e.g., transparency, reflectivity, color, etc.), taste properties (e.g., tastes such as "banana," "sour," "hot," etc.), storage stability, stability at particular pH levels, biodegradability, toxicity, industrial applicability, etc.
[0099] According to another aspect of the present disclosure, the machine-learned models described herein can be used in active learning techniques to narrow down a broad range of candidates to a smaller set of molecules or mixtures that are then manually evaluated. According to another aspect of the present disclosure, the systems and methods can enable the synthesis of molecules and / or mixtures with specific properties in an iterative design-test-improve process. For example, based on predictive data from the machine-learned models, molecules or mixtures can be suggested for development. The molecules or mixtures can then be synthesized and then specially tested. Feedback from the testing can then be fed back into the design phase to refine the molecules to better achieve desired properties, etc.
[0100] The methods, architecture, motivations, and practices utilized for molecular property prediction can be adopted or utilized for other initial predictions and can be utilized for overall mixture property prediction.
[0101] In some implementations, some property predictions may be determined based on the initially determined property predictions. Secondarily determined property predictions may be determined by utilizing known transfer characteristics and unlearned generic descriptors (e.g., SMILES strings, Morgan Fingerprints, Dragon Descriptors, etc.). These descriptors are typically intended to "characterize" molecules rather than convey complex structural interrelationships. For example, some existing approaches characterize or represent molecules with generic heuristic features such as Morgan Fingerprints or Dragon Descriptors. However, generic characterization strategies often fail to highlight important information relevant to a specific task, such as predicting olfactory or other sensory properties of a particular species of molecule. For example, Morgan Fingerprints are typically designed for "searching" for similar molecules. Morgan Fingerprints do not typically include the spatial arrangement of molecules. Still, while this information may be useful, it may be insufficient for some design cases, such as in the case of olfaction, which may benefit from spatial understanding. Nevertheless, a scratch-trained model with a small amount of available training data is unlikely to beat a Morgan Fingerprint model.
[0102] Another existing approach is physics-based modeling of sensory properties. For example, physics-based modeling may include computational modeling of sensory (e.g., olfactory) receptors or sensory-related (e.g., olfaction-related) proteins. For example, with a computational model of an olfactory receptor target, it is possible to perform high-throughput docking screens to find molecular candidates for a task of interest. However, this can be complicated for certain tasks, as modeling all possible interactions for all candidates can be computationally expensive. Furthermore, physics-based modeling of sensory properties may require explicit knowledge of the task at hand, such as the physical structure of the receptor, its binding pocket, and the location of the chemical ligand in that pocket, which may not be readily available. Furthermore, while some properties of molecules (e.g., pharmaceutical properties, material properties) can be easily learned, some sensory / sensory properties, especially sensory properties (e.g., olfactory properties), can be difficult to predict. This can be further complicated by the fact that the base of a particular scent chemical, such as ethanol, plastics, shampoos, soaps, fabrics, etc., can affect the perceived odor of the chemical. For example, the same chemical may be perceived differently in an ethanol base compared to, for example, a soap base. Thus, a chemical that has a large amount of available training data in one base may have only a limited amount of data in another base.
[0103] For example, in the field of insect repellents, some potential repellents may act as antagonists or secondary inhibitors, making it computationally expensive to model each possible interaction. Furthermore, for many sensory receptors, physical structures may not be available, making traditional docking simulations impossible. For example, from the perspective of insect repellent screening, existing methods used to predict chemical properties include simulating the docking of specific molecules in receptor pockets via detailed molecular dynamics simulations or binding mode predictions. However, these methods require expensive or difficult-to-obtain prior data to work in new domains, such as the crystal structure of the specific receptor to bind. This approach is often not possible or overly complicated, since perception (e.g., smell, taste) is the result of the cooperative activation of hundreds of different receptors, and the crystal structures of only a few receptors involved in chemical perception are known.
[0104] Exemplary aspects of the present disclosure may provide solutions to these and other challenges. According to aspects of the present disclosure, a machine-learned sensory prediction model may be trained on a first sensory prediction task and used to output predictions associated with a second sensory prediction task. As an example, the first sensory prediction task may be a broader sensory prediction task than the second sensory prediction task. For example, the model may be trained on a broad task and transferred to a narrower task. As an example, the first task may be a broad trait task and the second task may be a specific trait task (e.g., olfaction). Additionally and / or alternatively, the first sensory prediction task may be a task for which a larger amount of training data is available than the second sensory prediction task. Additionally and / or alternatively, the first sensory prediction task may be associated with a first species and the second sensory prediction task may be associated with a second species. As an example, the first sensory prediction task may be a human olfaction task. Additionally and / or alternatively, the second sensory prediction task may be a pest control task, such as a mosquito repellent task.
[0105] As an example, a sensory embedding model can be trained to generate a sensory embedding for a first sensory prediction task. The sensory embedding can be learned from a first sensory prediction task, such as a larger available dataset, such that the sensory embedding is specific to the first prediction task (e.g., a broader task). However, despite being trained for the first prediction task, in accordance with exemplary aspects of the present disclosure, it is recognized that the sensory embedding can obtain information useful for other (e.g., narrower) sensory prediction tasks. Furthermore, the sensory embedding can be transferred, fine-tuned, or otherwise modified to generate accurate predictions in another domain of a second sensory prediction task where less data is available than the first sensory prediction task, such as machine learning or tasks where accurate predictions are otherwise difficult and / or impossible.
[0106] As an example, the sensory embedding model can be trained in parallel with the first prediction task model. The sensory embedding model and the first prediction task model can be trained using the (e.g., labeled) first prediction task training data of the first prediction task. As an example, the sensory embedding model can be trained to generate sensory embeddings for the first prediction task. These sensory embeddings can obtain information useful for the second prediction task. After training the sensory embedding model with the first prediction task model based on the first prediction task training data, the sensory embedding model can be used with the second prediction task model to output predictions associated with the second prediction task. In some cases, the sensory embedding model can be further refined, fine-tuned, or otherwise continuously trained on the second prediction task training data associated with the second prediction task. In some implementations, the model may be trained at a lower training rate on the second prediction task than on the first prediction task to prevent information learned from the first prediction task from becoming unlearned intuitively. In some implementations, the amount of training data for the second prediction task may be less than the amount of training data for the first prediction task, such as when less data is available for the second prediction task than for the first prediction task.
[0107] The machine-learned models can be trained using training data including descriptions of molecules and / or mixtures (e.g., structural descriptions of molecules, graph-based descriptions of chemical structures of molecules, etc.) for a first sensory prediction task, such as molecules labeled (e.g., manually by an expert) with descriptions of sensory properties (e.g., olfactory properties) assessed for the molecules (e.g., textual descriptions of odor categories such as "sweet," "pine-like," "pear," "rotten," etc.). For example, these descriptions of olfactory molecules can be related to, for example, human perception. These models can then be used for a second sensory prediction task that is different from the first sensory prediction task. For example, the second sensory prediction task can be related to non-human perception. For example, in some embodiments, the models are transferred across sensory properties of molecules of different species.
[0108] In this way, models trained on large datasets can be transferred to tasks with smaller datasets while still achieving high predictive performance. In particular, it has been observed that sensory embeddings can significantly improve the quality of predictions when transferring learning across species for sensory (e.g., olfactory) prediction tasks. These sensory embeddings can go beyond intra-domain transfer learning to improve performance of even different natures, such as cross-species perception. This is unexpected, especially in the chemical field. For example, sensory embeddings can be taken directly as inputs for a second prediction task model. The sensory embedding model can then be fine-tuned and trained on the second sensory prediction task. Unexpectedly, the second sensory prediction task and the first sensory prediction task do not need to be overly similar. For example, prediction tasks with sufficient distinction (e.g., cross-species, different domains, etc.) can still find benefits according to exemplary aspects of the present disclosure.
[0109] Accordingly, some exemplary aspects of the present disclosure are directed to proposing the use of neural networks, such as graph neural networks, for olfaction, taste, and / or other sensory modeling across different domains, such as quantitative structure-odor relationship (QSOR) modeling. Graph neural networks can represent spatial information that may be important for modeling olfaction and other sensory senses. Exemplary implementations of the systems and methods described herein perform significantly better than conventional methods on novel datasets labeled by olfaction experts. Furthermore, sensory embeddings learned from graph neural networks obtain meaningful odor spatial representations that represent the underlying relationships between structures and odors. These learned sensory embeddings may be unexpectedly applied to domains other than the domain in which the model used to generate the sensory embeddings is learned. For example, models trained on human sensory perception data may unexpectedly achieve desirable results outside the human sensory perception domain, such as other species' perceptions and / or other domains. For example, the use of graph neural networks can provide models with spatial understanding that is beneficial for sensory modeling applications.
[0110] In some implementations, the predictions of the first prediction task and / or the second prediction task may indicate whether the molecule has a particular desired sensory property (e.g., the perception of a target scent, etc.). In some implementations, the prediction data may include one or more types of information related to the predicted sensory property (e.g., olfactory property) of the molecule. For example, the prediction data of a molecule may result in the classification of the molecule into one sensory property (e.g., olfactory property) class and / or into multiple sensory property (e.g., olfactory property) classes. In some cases, the classes may include human (e.g., expert) provided text labels (e.g., sour, cherry, pine-like, etc.). In some cases, the classes may include non-textual representations of the scent / odor, such as a position in a scent continuum. In some cases, the prediction data of a molecule may include an intensity value representing the intensity of the predicted scent / odor. In some cases, the prediction data may include a confidence value associated with the predicted olfactory sensory property. As another example, in some implementations, the prediction data may describe how well the molecule will perform in a particular task (e.g., a pest control task).
[0111] In addition to or instead of a specific classification of molecules, the prediction data can include a numerical sensory embedding that allows for similarity searching, clustering, or other comparison between two or more molecules based on a distance measure between the two or more sensory embeddings. For example, in some embodiments, the machine-learned model can be trained to output embeddings that can be used to measure similarity by training the machine-learned model using a triplet training scheme. In this scheme, the model is trained to output sensory embeddings that are closer in the sensory embedding space for pairs of similar chemical structures (e.g., anchor examples and positive examples) and sensory embeddings that are farther away in the sensory embedding space for pairs of dissimilar chemical structures (e.g., anchor and negative examples). According to exemplary aspects of the present disclosure, these output sensory embeddings can also be used in different tasks, such as heterogeneous tasks.
[0112] In another aspect of the present disclosure, training data including a plurality of known molecules can be obtained to train one or more machine-learned models (e.g., graph convolutional neural networks, other types of machine-learned models) to provide predictions of sensory properties (e.g., olfactory properties) of the molecules. For example, in some embodiments, the machine-learned models can be trained using one or more datasets of molecules, where the dataset includes, for each molecule, a chemical structure and a textual description of the sensory properties (e.g., a description of the odor of the molecule provided by a human expert, etc.). As an example, the training data can be obtained from publicly available data, such as, for example, a publicly available list of chemical structures and their corresponding odors. In some embodiments, due to the fact that some sensory properties are rare, steps can be taken to balance common sensory properties with rare sensory properties when training the machine-learned model(s). According to an exemplary aspect of the present disclosure, training data can be provided for a first sensory prediction task, where the training data is more widely available than the second sensory prediction task that is the overall purpose of the model. The model may then be retrained for the second sensory prediction task with a (limited) amount of training data for the second sensory prediction task, and / or may be used as is for the second sensory prediction task without further training.
[0113] Further, in some implementations, the systems and methods can provide an indication of how changes in molecular structure may affect predicted sensory properties (e.g., second prediction task). For example, the systems and methods can provide an indication of how changes in molecular structure may affect the intensity of a particular sensory property, how devastating the effect of changes in molecular structure is on the desired sensory property, etc. In some embodiments, the systems and methods can provide for adding and / or removing one or more atoms and / or groups of atoms from the structure of the molecule to determine the effect of such addition / removal on one or more desired sensory properties. For example, various modifications to the chemical structure can be iteratively performed and then the results evaluated to understand how such modifications affect the sensory properties of the molecule. As yet another example, the gradient of the classification function of the machine-learned model can be evaluated (e.g., with respect to a particular label) at each node and / or edge of the input graph (e.g., via backpropagation through the machine-learned model) to generate a sensitivity map (e.g., indicating how important each node and / or edge of the input graph is to the output of such particular label). Further, in some embodiments, a graph of a subject can be taken, similar graphs can be sampled by adding noise to the graph, and then an average of the resulting sensitivity maps for each sampled graph can be taken as the sensitivity map of the subject graph. Similar techniques can be performed to determine perceptual differences between different molecular structures.
[0114] Additionally, the systems and methods of the present disclosure can provide for interpretation and / or visualization of which aspects of a molecular structure contribute most to a predicted sensory quality (e.g., for a second prediction task). For example, in some embodiments, a heat map can be generated to overlay a molecular structure that provides an indication of which parts of the structure of the molecule are most important to the sensory properties of the molecule and / or which parts of the structure of the molecule are less important to the sensory properties of the molecule. In some implementations, data showing how changes in the molecular structure affect olfaction can be used to generate a visualization of how the structure contributes to a predicted olfactory quality. For example, as described above, iterative changes to the structure of a molecule (e.g., knockdown techniques, etc.) and their corresponding results can be used to evaluate which parts of a chemical structure contribute most to olfaction. As another example, as described above, a gradient technique can be used to generate a sensitivity map of a chemical structure, which can then be used to generate a visualization (e.g., in the form of a heat map).
[0115] The machine-learned model(s) may be trained to generate predictions of chemical structures of molecules or chemical formulations of mixtures that provide one or more desired sensory properties (e.g., generating chemical structures of molecules that produce a particular scent quality, etc.). For example, in some implementations, an iterative search may be performed to identify proposed molecule(s) or mixtures that are predicted to exhibit one or more desired sensory properties (e.g., a target scent quality, intensity, etc.). For example, the iterative search may suggest a number of candidate chemical structures of molecules or chemical formulations of mixtures that can be evaluated by the machine-learned model(s). In one example, the candidate molecular structures may be generated through an evolutionary or genetic process. As another example, the candidate molecular structures may be generated by a reinforcement learning agent (e.g., a recurrent neural network) that seeks to learn a policy that maximizes a reward that is a function of whether the generated candidate molecular structures exhibit one or more desired sensory properties. According to an exemplary aspect of the present disclosure, this sensory property analysis may be associated with a second sensory prediction task that is different from the first sensory prediction task.
[0116] The systems and methods can enable prediction, identification, and / or optimization of desired sensory properties (e.g., olfactory properties) as well as other properties associated with molecular structures. For example, the machine-learned model(s) can predict or identify properties of molecular structures, such as optical properties (e.g., transparency, reflectivity, color, etc.), olfactory properties (e.g., aroma, such as aroma reminiscent of fruit, flower, etc.), taste properties (e.g., taste, such as "banana", "sour", "hot", etc.), storage stability, stability at a particular pH level, biodegradability, toxicity, industrial applicability, etc., for a second sensory prediction task that is different from the first sensory prediction task for which the model(s) was previously trained.
[0117] In some embodiments, the machine-learned model can be used in active learning techniques to narrow down a wide range of candidates to a smaller set of molecules or mixtures, which are then manually evaluated. Alternatively and / or additionally, the system and method can enable the synthesis of molecules and / or mixtures with specific properties in an iterative design-test-improve process. For example, based on predictive data from the machine-learned model, a mixture can be suggested for development. The mixture can then be formulated and then special testing can be performed. Feedback from the testing can then be fed back to the design stage to refine the mixture to better achieve desired properties, etc. For example, the results of the testing can be used as training data to retrain the machine-learned model. After retraining, predictions from the model can then be used again to identify specific molecules or mixtures to test. Thus, iterative pipelines can be evaluated, such as using a model to select candidates, and then using the test results of the candidates to retrain the model.
[0118] For example, in one exemplary embodiment of the present disclosure, the model is trained using a large amount of readily available human perception data as training data. The model then moves to at least somewhat related chemistry problems, such as predicting whether a molecule or mixture will be a good mosquito repellent, discovering new flavor molecules, etc. The model (such as a neural network) can also be packaged into a standalone molecular embedding tool to generate representations focused on olfaction-related problems. These representations can be used to search for smells that smell similar to animals or that evoke similar behaviors. The embedding space described herein can also be useful as a codec for designing electronic odor perception systems (e.g., "electronic noses").
[0119] As another example, certain sensory properties may be desirable for animal attraction and / or repulsion tasks. For example, the first sensory prediction task may be a human sensory task, such as a human olfactory task, a human taste task, etc., based on the chemical structure of a molecule or mixture. The first sensory property may be a human sensory property, such as a human olfactory sensory property and / or a human taste sensory property. The second sensory prediction task may be a non-human sensory task, such as a related sensory task of another species. The second sensory prediction task may additionally and / or alternatively be or include the ability of the molecule as an attractant and / or repellent for a particular species. For example, the property may indicate the ability of the molecule in attracting a desired species (e.g., incorporation into an animal food product) or repelling an undesirable species (e.g., as an insect repellent).
[0120] For example, this may include pest control applications such as mosquito repellents, insecticides, etc. For example, mosquito repellents may help to repel mosquitoes and prevent bites that contribute to the transmission of viruses and diseases. For example, services or technologies related to the olfactory system of humans and / or animals may potentially find use in the systems and methods according to exemplary aspects of various embodiments. Exemplary embodiments may include approaches to find suitable odors for insect repellents or other pest control, such as, for example, mosquito repellents, crop health, livestock health, personal health, building / infrastructure health, and / or repellents for other suitable pests. For example, the systems and methods described herein may be useful for designing repellents, insecticides, attractants, etc. for target species of insects or other animals, even for animals for which little or no sensory perception data is available. As an example, the first sensory prediction task may be a sensory prediction task related to human senses, such as a human olfactory task predicting human olfactory perception labels based on molecular structure data. A second sensory prediction task might involve predicting the ability of a molecule to repel another species, such as mosquitoes.
[0121] As another example, the systems and methods according to exemplary embodiments of the present disclosure may be applicable to toxicology and / or other safety studies. As an example, the first sensory prediction task and / or the second sensory prediction task may be a toxicity prediction task. The sensory properties may be related to the toxicity of a chemical based on its chemical structure. As another example, the systems and methods according to exemplary embodiments of the present disclosure may be useful in transitioning to related olfactory tasks, such as finding molecules that smell similar to existing molecules but have different physical properties, such as color.
[0122] 2 illustrates a block diagram of an exemplary property prediction system 200 according to an exemplary embodiment of the present disclosure. In some implementations, the property prediction system 200 is trained to receive a set of input data 202, 204, 206, and 208 describing molecules in a mixture, and provides output data 216 including one or more property predictions describing predicted properties of the mixture as a result of receiving the input data 202, 204, 206, and 208. Thus, in some implementations, the property prediction system 200 can include one or more embedding model(s) 212 operable to generate molecular embeddings and a machine-learned predictive model 214 operable to generate one or more property predictions 216.
[0123] The property prediction system 200 can include a two-stage processing of input data to generate one or more property predictions 216. For example, in the illustrated system 200, the input data can include molecular data including molecular data 202, 204, 206, and 208 for each molecule in a mixture, where the molecular data can describe the N molecules, and the mixture data 210 describes the composition of the mixture of N molecules. The system 200 can process the molecular data using one or more embedding model(s) 212 to generate one or more embeddings that are processed by the machine-learned predictive model 214. In some implementations, the embedding model 212 can include a graph neural network (GNN) that generates one or more graphs. In some implementations, the molecular data can be processed such that each molecular data associated with each individual molecule can be processed individually, and each embedding can represent a single molecule.
[0124] The embeddings and mixture data 210 may be processed by a machine-learned predictive model 214 that generates one or more property predictions 216. The machine-learned predictive model 214 may include deep neural networks and / or various other architectures. Additionally, the property predictions 216 may include various predictions related to various properties associated with the mixture. For example, the property predictions 216 may include sensory property predictions, such as olfactory property predictions, that are subsequently used to create a fragrance.
[0125] Further, in this embodiment, the first molecule 202, the second molecule 204, the third molecule 206, ..., and the nth molecule 208 may be at the same or different concentrations in the theoretical mixture. The system may weight one or more embeddings based on the concentration of the molecules. The weighting may be completed by the embedding model 212, the machine-learned predictive model 214, and / or a third separate weighting model.
[0126] 3 illustrates a block diagram of an exemplary property prediction system 300 according to an exemplary embodiment of the present disclosure. The property prediction system 300 is similar to the property prediction system 200 of FIG. 2, except that the property prediction system 300 further includes three initial predictions.
[0127] More specifically, the illustrated system 300 includes three initial predictions that are made before an overall property prediction 330 is generated. For example, the system 300 may make individual molecule predictions 310, mixture composition property predictions 322, and mixture interaction property predictions 324, all of which may be factored into the overall property prediction 330.
[0128] The system 300 can begin by obtaining input data 310, which can include molecular data and mixture data describing a mixture with a set of molecules. The input data can be processed by a first model to generate molecule-specific predictions 310, which in some embodiments can be concentration-specific predictions. The concentration predictions 310 can be weighted based on the level of concentration, and predictions for various molecules can be pooled.
[0129] The output of the first model may then be processed by a second model 320, which may include two sub-models. The first sub-model may process the data and output composition-specific property predictions 322 related to the overall composition of the mixture. The second sub-model may process the data and output interaction-specific property predictions 324 related to the predicted interactions and / or predicted external interactions of the mixture.
[0130] The three initial predictions can be processed to generate an overall property prediction 330 based on each of the initial predictions to allow a better understanding of the mixture. For example, each individual molecule may have its own odor signature, but some molecular signatures may be more prevalent in a particular composition. Furthermore, the interaction properties of various molecules and sets of molecules may alter, enhance, or dilute certain odor signatures. Thus, each initial prediction provides insight into the odor, taste, etc. of the entire mixture.
[0131] 4 illustrates a block diagram of an exemplary property prediction request system 400 according to an exemplary embodiment of the present disclosure. In some implementations, the property prediction request system 400 is trained to receive a set of training data 442 and 444 describing known properties of individual molecules and known properties of mixture interactions, and determines and stores property predictions for one or more mixtures as a result of receiving the training data 442 and 444. Thus, in some implementations, the property prediction request system 400 can include a predictive computing system 402 operable to predict and store mixture properties.
[0132] The trait prediction request system 400 shown in FIG. 4 includes a prediction computing system 410, a requesting computing system 430, and a training computing system 440, which may communicate with each other to form the entire system 400.
[0133] In some implementations, the property prediction request system can rely on a trained predictive computing system 410 that can predict and store properties of mixtures to produce when later requested. Training the predictive computing system 410 can include the use of a training computing system 440 that can provide training data for training the machine-learned models 412 and 414 of the predictive computing system 410. For example, the training computing system 440 can have molecule training data 442 for training the first machine-learned model (e.g., an embedding model) 412 and mixture training data 444 for training the second machine-learned model (e.g., a deep neural network) 414. The training data can include known properties of various molecules, compositions, and interactions, and upon receiving the training data, can be stored in the predictive computing system for later reference. In some implementations, the training data can include a labeled training dataset that can include known properties of specific mixtures to complete ground truth training of the machine-learned models.
[0134] Additionally, the predictive computing system 410 can store the molecular data 416 and the mixture data 418 for reference, retraining, or data centralization. Alternatively and / or additionally, the molecular data 416 can be sampled to generate a database of mixture property predictions. Sampling can be random or influenced based on known molecular properties, molecular categories, and / or molecular abundances. The molecular data 416 and the mixture data 418 can be processed by the first machine-learned model 410 and the second machine-learned model to generate mixture property predictions that are stored 420 by the predictive computing system.
[0135] The stored data 420 may then be searchable or accessible via communication between the predictive computing system and the requesting computing system 430. The requesting computing system 430 may include a user interface 434 for a user to input a search query or request related to a particular mixture or a particular property. In response to the input, the requesting computing system 430 may generate a request 432 that may be sent to the predictive computing system 410 to search or screen the stored data to obtain and provide one or more results. The one or more results may then be returned to the requesting computing system, which may display the one or more results to the user via a user interface. In some implementations, the results may be one or more mixtures including property predictions associated with or matching the search query / request. In some implementations, the results may be provided as a mixture property profile including the mixtures and their respective property predictions.
[0136] 5 illustrates a block diagram of an exemplary mixture property profile 500 according to an exemplary embodiment of the present disclosure. In some implementations, the mixture property profile 500 is trained to receive and store property predictions for each mixture for property screening or retrieval. Thus, in some implementations, the mixture property profile 500 can include various property predictions that describe the predicted properties of the mixture.
[0137] 5 includes a grid of various property categories that can be filled with property predictions, known properties, or a mixture of known and predicted properties. In some embodiments, the mixture property profile 500 can include a reason for the property prediction including an initial prediction related to the composition of the mixture, the predicted property, a graphical depiction of the mixture or molecules in the mixture, and / or the molecules in the mixture, and / or interactions within the mixture.
[0138] Some example properties displayed in the mixture property profile 500 may include odor property 504, taste property 506, color property 508, viscosity property 510, lubricant property 512, thermal property 514, energy property 516, medicinal property 518, stability property 520, catalytic property 522, adhesive property 524, and miscellaneous property 526.
[0139] Each characteristic may be searchable to obtain a mixture with desired characteristics upon request or query. Additionally, each characteristic may provide desirable insights for use in a variety of different areas, including consumer, industrial, and the like. For example, odor characteristics 504 may include odor quality characteristics and odor intensity characteristics, which may be utilized to create fragrances, perfumes, candles, and the like. Taste characteristics 506 may be utilized to create artificial flavors for candies, vitamins, or other consumables. Prediction of characteristics may be based at least in part on predicted receptor interactions and activation. Other characteristics may be used for marketing of the product, such as color characteristics 508, which may be used to predict the color of the mixture or may include coloring characteristics. Coloring characteristics may be predicted to determine whether the mixture has the potential to color other products. Viscosity characteristics 510 may be another characteristic predicted and stored.
[0140] Other property predictions can be relevant for industrial applications such as yielding lubricant properties 512 for machine dynamics, and energetic properties 516 can be used to make better batteries. Pharmaceuticals can also be improved or formulated based on knowledge gained from these property predictions.
[0141] FIG. 9A illustrates an exemplary evolutionary approach 900 that can be used to generate a database of new mixtures with predicted properties. The proposed mixtures can have molecular and mixture data 902 for each respective proposed mixture. The molecular and mixture data 902 can be processed by a machine learning property prediction system 904 to generate predicted properties 906 for the proposed mixtures. The predicted properties 906 can then be processed by an objective function 908 to determine whether an addition to a corpus of top performers 910 should be made or discarded. Random mutations can occur and the process can begin again. The evolutionary approach 900 can be useful in generating a large database of useful mixtures available for screening by human practitioners for use in various products and industries.
[0142] 9B illustrates an exemplary reinforcement learning approach 950 that can be used to optimize a model. Similar to the evolutionary approach 900, the reinforcement learning approach 950 can start with molecular and mixture data 902 of a proposed mixture that is processed by a machine learning property prediction system to generate a predicted property 906. The predicted property 906 can then be processed by an objective function 912 to provide an output to a machine learning controller 914 to provide a suggestion to the system. In some implementations, the machine learning controller can include a recurrent neural network. In some implementations, the reinforcement learning approach 950 can help refine parameters of the machine learning models disclosed herein.
[0143] Exemplary Methods 6 illustrates a flowchart diagram of an exemplary method performed according to an exemplary embodiment of the present disclosure. Although FIG. 6 illustrates steps performed in a particular order for purposes of illustration and discussion, the methods of the present disclosure are not limited to the order or arrangement specifically illustrated. Various steps of method 600 may be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.
[0144] At 602, the computing system may obtain molecular data and mixture data. The molecular data may be data describing one or more molecules of the mixture, and the mixture data may describe the mixture. In some embodiments, the molecular data may include respective molecular data for each of the plurality of molecules, and the mixture data may describe the chemical composition of the mixture. The data may be obtained by manually entered data or automatically sampled data. In some embodiments, the molecular data and the mixture data may be obtained from a server. In some embodiments, the mixture data may include the concentration of each molecule of the mixture.
[0145] At 604, the computing system can process the molecular data with the embedding model to generate one or more embeddings. The respective molecular data for each of the plurality of molecules can be processed with the embedding model to generate a respective embedding for each molecule. In some implementations, the embedding model can include a graph neural network that generates embeddings of one or more graphs. The embeddings can include embedding data that describes individual molecular properties.
[0146] At 606, the computing system can process the embeddings and mixture data using a machine-learned predictive model, which can include a deep neural network and can also include a weighting model that can weight and pool the embeddings based on their respective molecular concentrations.
[0147] At 608, the computing system can generate one or more property predictions. The one or more property predictions can be based at least in part on one or more of the embeddings and the mixture data. Furthermore, the predictions can be based on properties of individual molecules, concentrations of molecules in the mixture, composition of the mixture, and interaction properties of the mixture. In some embodiments, the predictions can be sensory predictions, energy predictions, stability predictions, and / or thermal predictions.
[0148] At 610, the computing system can store one or more property predictions. The property predictions can be stored in a searchable database to allow easy retrieval of mixtures and properties.
[0149] 7 illustrates a flowchart diagram of an exemplary method performed according to an exemplary embodiment of the present disclosure. Although FIG. 7 illustrates steps performed in a particular order for purposes of illustration and discussion, the methods of the present disclosure are not limited to the order or arrangement specifically illustrated. Various steps of method 700 may be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.
[0150] At 702, the computing system may acquire molecular data and mixture data. In some embodiments, the molecular data may describe a plurality of molecules of the mixture and the mixture data may describe the mixture. The molecular data and the mixture data may be acquired separately or simultaneously.
[0151] At 704, the computing system may process the molecular data with the embedding model to generate embeddings. The embedding model may be a graph embedding model, and the embeddings may be graph embeddings. In some implementations, the graph embeddings may be weighted and pooled to generate a graph of graphs. In some implementations, the respective molecular data for each of the plurality of molecules may be processed with the embedding model as a molecule-specific set to generate a respective embedding for each molecule.
[0152] At 706, the computing system can process the embedding data and the mixture data using a machine-learned predictive model to generate one or more property predictions. The property predictions can include predictions for various mixture properties and can be used in a variety of fields and industries.
[0153] The computing system can store one or more property predictions at 708. The property predictions can be stored in a searchable database for easy access to the information.
[0154] At 710, the computing system may obtain a request for a mixture with a requested characteristic and determine one or more characteristic predictions that constitute the requested characteristic. The request may be a formal request or may be a search query entered into a user interface. In some implementations, the determining may include determining whether a predicted characteristic matches the requested characteristic or is associated with the search query.
[0155] At 712, the computing system may provide the mixture data to the requesting computing device. The requesting computing device may receive the mixture data in a variety of formats, including textual data, graphical data, etc. In some implementations, the mixture data may be provided with a mixture property profile indicating a property prediction for each mixture.
[0156] 8 illustrates a flowchart diagram of an exemplary method performed according to an exemplary embodiment of the present disclosure. Although FIG. 8 illustrates steps performed in a particular order for purposes of illustration and discussion, the methods of the present disclosure are not limited to the order or arrangement specifically illustrated. Various steps of method 800 may be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.
[0157] At 802, a computing system can obtain molecular data and mixture data.
[0158] At 804, the computing system may process the molecular data with the first model to generate molecular property predictions. In some implementations, the molecular property predictions may be embedded before being processed by the second model.
[0159] At 806, the computing system can process the molecular property predictions and the mixture data with a second model to generate a mixture property prediction. The mixture property prediction can be based at least in part on the molecular property predictions and the concentrations of the one or more molecules.
[0160] At 808, the computing system can generate a predicted property profile for the mixture. The property profile can be organized data including the mixture, property predictions for the mixture, and other data necessary for application of the mixture in a desired field.
[0161] At 810, the computing system can store the predicted trait profiles in a searchable database that may be enabled by other applications or may be a stand-alone searchable database with a dedicated interface.
[0162] Additional Disclosures The technology described herein refers to servers, databases, software applications, and other computer-based systems, as well as actions performed and information sent to and from such systems. The inherent flexibility of computer-based systems allows for a wide variety of configurations, combinations, and divisions of tasks and functionality among components. For example, the processes described herein may be implemented using a single device or component, or multiple devices or components working in combination. Databases and applications may be implemented on a single system or distributed across multiple systems. Distributed components may operate sequentially or in parallel.
[0163] Although the subject matter of the present invention has been described in detail with respect to various specific exemplary embodiments thereof, each example is provided for illustrative purposes and not to limit the present disclosure. Those skilled in the art, upon understanding the foregoing, can easily create modifications, variations, and equivalents to such embodiments. Thus, the present disclosure is not intended to preclude inclusion of such modifications, variations, and / or additions to the subject matter as would be readily apparent to one skilled in the art. For example, features illustrated or described as part of one embodiment can be used with another embodiment to obtain yet another embodiment. Thus, the present disclosure is intended to cover such modifications, variations, and equivalents.
Claims
1. 1. A computer-implemented method for mixture property prediction, comprising: obtaining, by a computing system including one or more computing devices, respective molecular data for each of a plurality of molecules and mixture data associated with a mixture of the plurality of molecules; processing, by the computing device, the respective molecular data for each of the plurality of molecules with a machine-learned embedding model to generate a respective embedding for each molecule; processing, by the computing system, the embeddings and the mixture data with a predictive model to generate one or more property predictions for the mixture of the plurality of molecules, the one or more property predictions being based at least in part on the embeddings and the mixture data; storing, by the computing system, the one or more property predictions; A method comprising:
2. The method of claim 1 , wherein the mixture data describes the respective concentrations of each molecule in the mixture.
3. The method of any one of claims 1 to 2, wherein the mixture data describes a composition of the mixture.
4. The method of any one of claims 1 to 3, wherein the predictive model comprises a deep neural network.
5. The method of claim 1 , wherein the machine-learned embedding model comprises a machine-learned graph neural network.
6. The method of any one of claims 1 to 5, wherein the predictive model comprises a characteristic specific model configured to generate predictions for specific properties.
7. The method of any one of claims 1 to 6, wherein the one or more property predictions are based at least in part on binding energies of one or more molecules of the plurality of molecules.
8. The method of any one of claims 1 to 7, wherein the one or more property predictions comprise one or more sensory property predictions.
9. The method of any one of claims 1 to 8, wherein the one or more property predictions comprises an olfactory prediction.
10. The method of any one of claims 1 to 9, wherein the one or more property predictions comprises a catalyst property prediction.
11. The method of any one of claims 1 to 10, wherein the one or more property predictions include an energy property prediction.
12. The method of any one of claims 1 to 11, wherein the one or more property predictions comprises target-to-target surfactant property predictions.
13. The method of any one of claims 1 to 12, wherein the one or more property predictions comprises a pharmaceutical property prediction.
14. The method of any one of claims 1 to 13, wherein the one or more property predictions include a thermal property prediction.
15. 15. The method of any one of claims 1 to 14, wherein the predictive model comprises a weighting model configured to weight and pool the embeddings based on the mixture data, the mixture data comprising concentration data associated with the plurality of molecules of the mixture.
16. obtaining, by the computing system, a request for a chemical mixture having requested properties from a requesting computing device; determining, by the computing system, that the one or more property predictions satisfy the requested property; providing, by said computing system, said mixture data to said requesting computing device; The method of any one of claims 1 to 15, further comprising:
17. The method of any one of claims 1 to 16, wherein the one or more property predictions are based at least in part on molecular interaction properties.
18. The method of any one of claims 1 to 17, wherein the one or more property predictions are based at least in part on receptor activation data.
19. 1. A computing system comprising: one or more processors; one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations; Including, The operation, acquiring molecular data for each of a plurality of molecules and mixture data relating to a mixture of the plurality of molecules, the mixture data including a respective concentration of each molecule of the plurality of molecules; processing the respective molecular data with an embedding model for each of the plurality of molecules to generate a respective embedding for each molecule; processing the embeddings and the mixture data with a machine-learned predictive model to generate one or more property predictions, the one or more property predictions based at least in part on the embeddings and the mixture data; and storing the one or more property predictions; and A computing system comprising:
20. One or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more processors, cause a computing system to perform operations, the operations including: obtaining respective molecular data for a plurality of molecules and mixture data relating to a mixture of said plurality of molecules; processing the respective molecular data with an embedding model for each of the plurality of molecules to generate a respective embedding for each molecule; processing the embeddings and the mixture data with a machine-learned predictive model to generate one or more property predictions, the one or more property predictions based at least in part on the embeddings and the mixture data; and storing the one or more property predictions; and [0023] In one or more non-transitory computer readable media,
