Three-dimensional molecular generation in latent voxelization space

Abstract

The voxelized representation of an input molecule can be encoded to generate an embedding representation of the input molecule that has fewer features than the voxelized representation of the input molecule. A molecular design computation model can be applied to update the embedding representation of the input molecule. The molecular design computation model can be trained to approximate the data distribution of molecules exhibiting one or more desired properties by taking corrupted embedding representations of voxelized representations of sample molecules exhibiting one or more desired properties as input and reconstructing the embedding representations of the voxelized representations of the sample molecules. The molecular design computation model can update the embedding representation of the input molecule to increase the likelihood that the resulting updated embedding representation falls within the data distribution. At a minimum, the voxelized representation of the output molecule can be generated by decoding the resulting updated embedding representation.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims priority to U.S. Provisional Application No. 63 / 502,529, filed May 16, 2023, entitled "THREE - DIMENSIONAL MOLECULE GENERATION BY DENOISING VOXEL GRIDS"; U.S. Provisional Application No. 63 / 586,263, filed September 28, 2023, entitled "THREE - DIMENSIONAL MOLECULE GENERATION BY DENOISING VOXEL GRIDS"; and U.S. Provisional Application No. 63 / 623,062, filed January 19, 2024, entitled "THREE - DIMENSIONAL MOLECULE GENERATION BY DENOISING VOXEL GRIDS", the disclosures of which are hereby incorporated by reference in their entireties.

[0002] Technical Field The subject matter described herein generally relates to generative artificial intelligence, and more specifically, to machine - learning - enabled techniques for generating representations of three - dimensional molecules in discrete and latent voxelized spaces.

Background Art

[0003] Introduction A molecule is a group of two or more atoms held together by chemical bonds. A molecule forms the smallest recognizable unit of a pure substance that can be divided while still retaining its composition and chemical properties. An example of a molecule is a small molecule, which is a low-weight compound with a molecular weight of approximately 100 to 1000 daltons. Small molecule therapeutics, which modulate biochemical processes to diagnose, treat, and prevent various diseases, form the basis of modern pharmacology due to several compelling advantages. For example, small molecule drugs can penetrate cell membranes to reach intracellular targets. Furthermore, small molecule drugs are adaptable to a wide variety of therapeutic applications. For instance, small molecule drugs can be formulated as pills and capsules, intravenous or subcutaneous injections, inhalants, or suppositories. The development of small molecule drugs can be further extended to modulating various pharmacokinetic properties, including release, absorption, dispersion, metabolism, potency, efficacy, phenotypic effects, and excretion.

[0004] In contrast, large molecules (also known as biopharmaceuticals, biologics, or biological drugs) can have molecular weights ranging from approximately 3,000 to 150,000 daltons. Large molecule drugs are often derivatives of native human proteins that regulate many essential cellular functions such as enzymatic reactions, molecular transport, regulation and execution of several biological pathways, cell proliferation, growth, nutrient uptake, morphology, motility, and intercellular communication. A single large molecule commonly has more than 1,300 amino acid residues linked by peptide bonds to form one or more polypeptides. Due to their size and complexity, large molecule drugs are recombinantly produced by engineered cells, rather than being chemically synthesized like most small molecule drugs. Furthermore, large molecule therapeutics are usually delivered by injection or infusion due to the ineffectiveness of oral administration. The development of large molecule drugs can involve designing sequences of one or more amino acid residues that can bind to a target (e.g., protein, nucleic acid, etc.) with sufficient specificity and that do not possess undesirable properties such as immunogenicity, self-association, or instability. [Overview of the project]

[0005] Systems, methods, and products, including computer program products, are provided for generating three-dimensional molecules in latent voxelization space. In one embodiment, a system for machine learning-enabled three-dimensional molecule generation is provided. The system may include at least one processor and at least one memory. The at least one memory may contain program code that, when executed by at least one processor, brings about an action. The operation may include encoding a voxelized representation of an input molecule to generate an embedding representation of the input molecule having fewer features than the voxelized representation of the input molecule, and applying a molecular design computation model to update the embedding representation of the input molecule, wherein the molecular design computation model is trained to approximate a data distribution of molecules exhibiting one or more desirable properties by taking a corrupted embedding representation of a voxelized representation of a sample molecule exhibiting one or more desired properties as input, and reconstructing the embedding representation of the voxelized representation of the sample molecule from the corrupted embedding representation, and the molecular design computation model updates and applies the embedding representation of the input molecule to increase the likelihood that the resulting updated embedding representation is in the data distribution, and at least generating a voxelized representation of an output molecule by decoding the resulting updated embedding representation.

[0006] In another embodiment, a method for machine learning-enabled three-dimensional molecule generation is provided. This method may include: encoding a voxelized representation of an input molecule to generate an embedding representation of the input molecule having fewer features than the voxelized representation of the input molecule; and applying a molecular design computation model to update the embedding representation of the input molecule, wherein the molecular design computation model is trained to approximate a data distribution of molecules exhibiting one or more desirable properties by taking a corrupted embedding representation of a voxelized representation of a sample molecule exhibiting one or more desired properties as input, and reconstructing the embedding representation of the voxelized representation of the sample molecule from the corrupted embedding representation; and updating and applying the embedding representation of the input molecule to increase the likelihood that the resulting updated embedding representation is in the data distribution; and generating a voxelized representation of an output molecule by decoding the resulting updated embedding representation.

[0007] In another embodiment, a computer program product for machine learning-enabled three-dimensional molecule generation is provided. The computer program product may include a non-temporary computer-readable medium that stores instructions that cause an action when executed by at least one data processor. The action may include encoding a voxelized representation of an input molecule to generate an embedding representation of the input molecule having fewer features than the voxelized representation of the input molecule, and applying a molecular design calculation model to update the embedding representation of the input molecule, wherein the molecular design calculation model is trained to approximate a data distribution of molecules exhibiting one or more desirable properties by taking corrupted embedding representations of voxelized representations of sample molecules exhibiting one or more desired properties as input, and reconstructing the embedding representation of the voxelized representation of the sample molecules from the corrupted embedding representation, and the molecular design calculation model updates and applies the embedding representation of the input molecule to increase the likelihood that the resulting updated embedding representation is in the data distribution, and at least generating a voxelized representation of an output molecule by decoding the resulting updated embedding representation.

[0008] Some modifications may include, in any feasible combination, one or more of the features disclosed herein, including the following features:

[0009] In some variations, the data distribution may be a noisy data distribution occupied by noisy embedding representations of voxelized representations of molecules exhibiting one or more desirable properties. The voxelized representation of the output molecule may be further generated by denoising the noisy voxelized representation of the output molecule, which is produced by decoding the resulting updated embedding representation.

[0010] In some variations, a vector quantized variational autoencoder (VQ-VAE) may be applied to encode the voxelized representation of the input molecule and decode the resulting updated embedding representation.

[0011] In some variations, the molecular embedding representation may be a discrete latent embedding vector produced by quantizing the corresponding continuous latent embedding representation. The quantization may include matching the corresponding continuous latent embedding representation with a vector in the embedding representation's codebook by nearest neighbor search.

[0012] In some variations, the voxelized representation of the input molecule may be encoded by compressing multiple atomic density values, including the voxelized representation of the input molecule, such that the embedding representation of the input molecule contains fewer features than the voxelized representation of the input molecule.

[0013] In some variations, the voxelized representation of a molecule may include multiple voxels organized into a three-dimensional voxel grid. Each atom in the molecule may be represented as a continuous density across one or more voxels in the three-dimensional voxel grid.

[0014] In some variations, the continuity density of each atom in the molecule may be centered at the center of each atom. A first voxel located far from any atom in the molecule may be associated with a lower atomic density value than a second voxel located closer to the center of an atom in the molecule.

[0015] In some variations, each voxel in a three-dimensional voxel grid may be associated with a value that indicates the atomic density at its corresponding location.

[0016] In some variations, the voxelized representation of a molecule may include one or more channels. Each channel may correspond to a type of atom present in the molecule.

[0017] In some variations, the voxelized representation of a molecule may also represent the type and position of one or more atoms present in the molecule.

[0018] In some variations, the embedding representation of the input molecule may be updated based on a function parameterized by at least several parameters of the molecular design computation model. The function may output a value indicating the likelihood of the resulting updated embedding representation within the data distribution.

[0019] In some variations, the function may be a score function. The value output by the function may be a score indicating a local change in the density of the noisy data distribution at the location of each updated noisy embedding representation generated by updating the noisy embedding representation of the input molecule.

[0020] In some variations, the molecular design calculation model may update the embedding representation of an input molecule by at least: applying the molecular design calculation model to update the embedding representation of the input molecule, thereby generating a first updated embedding representation; applying the molecular design calculation model to update the embedding representation of the input molecule, thereby generating a second updated embedding representation; applying a function parameterized by several parameters of the molecular design calculation model to determine (i) a first value indicating a first local change in the density of the data distribution at a first location occupied by the first updated embedding representation, and (ii) a second value indicating a second local change in the density of the data distribution at a second location occupied by the second updated embedding representation; and applying the molecular design calculation model to further update the first updated embedding representation in place of the second updated embedding representation based on at least the first and second values.

[0021] In some variations, the molecular design calculation model may be applied to further update the first updated embedding representation until one or more criteria are met. These one or more criteria may include at least one of the following: (i) the update iterations for the embedding representation of the input molecule have been performed a threshold number of times; (ii) the first value of the first updated embedding representation satisfies one or more thresholds; and (iii) a threshold amount of output molecule has been generated.

[0022] In some variations, the molecular design calculation model may be applied to further modify the first updated embedding representation instead of the second updated embedding representation, based on the fact that at least the first and second values ​​indicate that the first updated embedding representation is more likely to be contained within the data distribution than the second updated embedding representation.

[0023] In some variations, the molecular design calculation model may be further applied to modify the first updated embedded representation instead of the second updated embedded representation, based at least on indicating that the first value and the second value are sampled from a high-density region of the data distribution rather than the second updated embedded representation.

[0024] In some variations, the voxelized representation of the output molecule may be converted into a one-dimensional representation and / or a two-dimensional representation of the output molecule.

[0025] In some variations, the voxelized representation of the output molecule may be converted by at least determining the positions of one or more atoms in the output molecule by detecting one or more peaks in a plurality of atomic density values including at least the voxelized representation of the output molecule, and at least determining one or more bonds based on the positions of the one or more atoms.

[0026] Systems, methods, and products, including computer program products, are provided for generating three-dimensional molecules in latent voxelization space. In one embodiment, a system for machine learning-enabled three-dimensional molecule generation is provided. The system may include at least one processor and at least one memory. The at least one memory may contain program code that, when executed by at least one processor, brings about an action. The operation may include generating a training dataset containing multiple training samples, wherein each training sample in the training dataset contains a corrupted embedding representation generated by adding at least noise to a noisy voxelized representation of a sample molecule exhibiting one or more desirable properties; training a molecular design computation model to approximate the data distribution of one or more molecules exhibiting one or more desirable properties based at least on the training dataset, wherein the training includes applying the molecular design computation model to reconstruct an uncorrupted embedding representation of a noisy voxelized representation of a sample molecule from a corrupted embedding representation of a noisy voxelized representation of a sample molecule; and optionally applying the molecular design computation model to generate an output molecule by at least denoising the embedding representation of the voxelized representation of an input molecule and decoding the resulting updated embedding representation to generate a voxelized representation of an output molecule.

[0027] In another aspect, a method for machine learning-enabled three-dimensional molecular generation is provided. The method includes generating a training dataset including a plurality of training samples, where each training sample in the training dataset includes a corrupted embedding representation generated by adding at least noise to a noisy voxelized representation of a sample molecule exhibiting one or more desirable properties; training a molecular design computational model to approximate a data distribution of molecules exhibiting one or more desirable properties, based at least on the training dataset, where the training includes applying the molecular design computational model to restore an uncorrupted embedding representation of the noisy voxelized representation of the sample molecule from the corrupted embedding representation of the noisy voxelized representation of the sample molecule; and optionally, applying the molecular design computational model to generate an output molecule by at least denoising an embedding representation of a voxelized representation of an input molecule and decoding the resulting updated embedding representation to generate a voxelized representation of the output molecule.

[0028] In another embodiment, a computer program product for machine learning-enabled three-dimensional molecule generation is provided. The computer program product may include a non-temporary computer-readable medium that stores instructions that cause an action when executed by at least one data processor. The action may include generating a training dataset containing a plurality of training samples, wherein each training sample in the training dataset includes a corrupted embedding representation generated by adding at least noise to a noisy voxelized representation of a sample molecule exhibiting one or more desirable properties; training a molecular design computation model to approximate the data distribution of molecules exhibiting one or more desirable properties based on at least the training dataset, wherein the training includes applying the molecular design computation model to reconstruct an uncorrupted embedding representation of a noisy voxelized representation of a sample molecule from a corrupted embedding representation of a noisy voxelized representation of a sample molecule; and optionally, applying the molecular design computation model to generate an output molecule by at least denoising the embedding representation of the voxelized representation of an input molecule and decoding the resulting updated embedding representation to generate a voxelized representation of an output molecule.

[0029] Some modifications may include, in any feasible combination, one or more of the features disclosed herein, including the following features:

[0030] In some variations, the noisy voxelized representation of the sample molecule may include multiple voxels organized into a three-dimensional voxel grid. Each atom in the sample molecule may be represented as a continuous density across one or more voxels in the three-dimensional voxel grid.

[0031] In some variations, the continuum density of each atom in the sample molecule may be centered at the center of each atom. A first voxel located far from any atom in the sample molecule is associated with a lower atomic density value than a second voxel located closer to the center of the atom in the sample molecule.

[0032] In some variations, each voxel in a three-dimensional voxel grid may be associated with a value that indicates the atomic density at its corresponding location.

[0033] In some variations, the noisy voxelized representation of the sample molecule may include one or more channels. Each channel may correspond to a type of atom present in the sample molecule.

[0034] In some variations, the noisy voxelized representation of the sample molecule may represent both the type and position of one or more atoms present in the sample molecule.

[0035] In some variations, training the molecular design computation model may involve tuning several parameters of the molecular design computation model to reduce the difference between the reconstructed embedding representation generated by the molecular design computation model and the undamaged embedding representation of the noisy voxelized representation of the sample molecule.

[0036] In some variations, multiple parameters of the molecular design calculation model may parameterize a function. The multiple parameters may be adjusted to output values ​​that indicate local changes in the density of the data distribution of molecules in which the function exhibits one or more desirable properties.

[0037] In some variations, the molecular design calculation model may denoise the embedding representation of the voxelized representation of the input molecule by updating at least one value in the embedding representation that represents multiple atomic density values ​​present in the voxelized representation of the input molecule.

[0038] In some variations, updating the atomic density values ​​of one or more voxels in at least one channel of the voxelized representation of the input molecule may correspond to updating at least one of the types and / or positions of one or more atoms present in the input molecule.

[0039] In some variations, the molecular design computation model may denoise the embedding representation of the voxelized representation of the input molecule over multiple iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling until one or more criteria are met.

[0040] In some variations, one or more criteria may include at least one of the following: (i) a gradient-based Markov chain Monte Carlo (MCMC) sampling iteration has been performed a threshold number of times; (ii) the resulting updated embedding representation has been sampled from a region having a threshold density; and (iii) a threshold output molecule has been generated.

[0041] In some variations, the molecular design calculation model may generate the voxelized representation of the output molecule by, at least, applying a first update to the embedding representation of the voxelized representation of the input molecule to generate a first updated embedding representation; applying a second update to the embedding representation of the voxelized representation of the input molecule to generate a second updated embedding representation; and, upon determining that the first updated embedding representation is sampled from a higher density region of the data distribution than the second updated embedding representation, further updating the first updated embedding representation instead of the second updated embedding representation.

[0042] In some variations, the voxelized representation of the output molecule may be converted to a one-dimensional and / or two-dimensional representation of the output molecule.

[0043] In some variations, training the molecular design computation model may include: applying the molecular design computation model with a first adjustment to generate a first restored embedding representation of a noisy voxelized representation of a sample molecule; determining a first mean squared error (MSE) that quantifies a first difference between the first restored embedding representation and an unbroken embedding representation of the noisy voxelized representation of the sample molecule; applying the molecular design computation model with a second adjustment to generate a second restored embedding representation of the noisy voxelized representation of the sample molecule; determining a second mean squared error (MSE) that quantifies a second difference between the second restored embedding representation and an unbroken embedding representation of the noisy voxelized representation of the sample molecule; and, upon determination that the first mean squared error (MSE) is smaller than the second mean squared error (MSE), further adjusting the molecular design computation model with the first adjustment instead of the second adjustment.

[0044] In some variations, the molecular design calculation model may be further refined until one or more criteria are met. These criteria may include at least one of the following: (i) the refining of the molecular design calculation model has been performed a threshold number of times, and (ii) a restored embedding representation exhibiting a threshold mean squared error (MSE) value has been generated.

[0045] In some variations, an autoencoder comprising an encoder and a decoder may be trained. Training the autoencoder may include training the encoder to encode a noisy voxelized representation of a sample molecule so that the decoder can reconstruct the voxelized representation of the sample molecule from an embedding representation resulting from a noisy voxelized representation of the sample molecule.

[0046] In some variations, the autoencoder may be a vector quantized variational autoencoder (VQ-VAE), in which the encoder generates a continuous latent embedding representation of the sample molecule, which is then quantized by matching it to a vector in the embedding representation's codebook through nearest neighbor search to obtain a discrete latent embedding representation for decoding by the decoder.

[0047] Embodiments of this subject matter include, but are not limited to, methods of operation as described herein, and articles comprising a tangibly embodied machine-readable medium capable of operating one or more machines (e.g., a computer) to produce operations that implement one or more of the features described herein. Similarly, computer systems are also described, which may include one or more processors and one or more memories coupled to one or more processors. A memory, which may include a non-temporary computer-readable or machine-readable storage medium, may contain, code, or store one or more programs causing one or more processors to perform one or more of the operations described herein. Computer implementations consistent with one or more embodiments of this subject matter may be implemented by one or more data processors residing in a single computing system or multiple computing systems. Such multiple computing systems may be connected and exchange data and / or commands or other instructions, etc., over one or more connections, including, for example, connections over a network (e.g., the Internet, a wireless wide area network, a local area network, a wide area network, a wired network, etc.), connections via direct connections between one or more of the multiple computing systems.

[0048] Details of one or more modifications of the subject matter described herein are described in the accompanying drawings and the following description. Other features and advantages of the subject matter described herein will become apparent from the description and drawings, as well as from the claims. Certain features of the subject matter currently disclosed are described for illustrative purposes in relation to the computational design of molecules, including drug molecules, but it should be readily understood that such features are not intended to be limiting. The claims following this disclosure define the scope of the subject matter protected. [Brief explanation of the drawing]

[0049] The accompanying drawings incorporated herein and constituting part of this specification illustrate specific aspects of the subject matter disclosed herein and, together with the description, help to illustrate some of the principles relating to the disclosed embodiments.

[0050] [Figure 1A] A system diagram illustrating an example of a molecular design system according to several exemplary embodiments.

[0051] [Figure 1B] A system diagram illustrating another example of a molecular design system by several exemplary embodiments.

[0052] [Figure 2] A flowchart illustrating an example of a process for generating machine learning correspondences for three-dimensional molecules in voxelized space, based on several exemplary embodiments.

[0053] [Figure 3A] A flowchart illustrating an example of the process for training a molecular design computation model to generate three-dimensional molecules in voxelized space, using several exemplary embodiments.

[0054] [Figure 3B] A flowchart illustrating an example of the process for applying a molecular design computation model to generate a three-dimensional molecule in voxelized space, using several exemplary embodiments.

[0055] [Figure 3C] A flowchart illustrating an example of the process for applying a molecular design computation model to generate a three-dimensional molecule in voxelized space, using several exemplary embodiments.

[0056] [Figure 4] A figure illustrating an example of a voxelized representation of a molecule by several exemplary embodiments.

[0057] [Figure 5A] A schematic diagram illustrating an example of a process for training a denoising engine to denoise noisy voxelized representations of molecules, based on several exemplary embodiments.

[0058] [Figure 5B] A schematic diagram illustrating an example of a walk-jump sampling method according to several exemplary embodiments.

[0059] [Figure 5C] A schematic diagram illustrating an example of the process by which a molecular design computation model generates a three-dimensional molecule by denoising a voxelized molecular representation, based on several exemplary embodiments.

[0060] [Figure 5D] A schematic diagram illustrating an example of a process for generating other molecular representations from a voxelized representation of a molecule, using several exemplary embodiments.

[0061] [Figure 6] A schematic diagram illustrating an example of the process for generating a voxelated representation of a molecule by having a molecular design computation model operate within a noisy latent voxelization space, based on several exemplary embodiments.

[0062] [Figure 7]A graph illustrating the effect of noise levels on the generation performance of molecular design calculation models in several exemplary embodiments.

[0063] [Figure 8] A schematic diagram illustrating the effect of the number of sampling iterations in Markov chain Monte Carlo (MCMC) sampling on the generation performance of molecular design computation models in several exemplary embodiments.

[0064] [Figure 9A] A figure illustrating examples of voxelized representations of molecules generated by molecular design computation models trained on the QM9 molecular dataset, according to several exemplary embodiments.

[0065] [Figure 9B] Figure showing examples of voxelized representations of molecules generated by molecular design computation models trained on molecular geometry ensemble (GEOM) drug datasets, in several exemplary embodiments.

[0066] [Figure 10A] A graph showing the cumulative distribution function (CDF) of strain energy for molecules in the QM9 molecular dataset, molecules generated by conventional generative models, and molecules generated by molecular design computation models trained on the QM9 molecular dataset, in several exemplary embodiments.

[0067] [Figure 10B] A graph showing the empirical distribution of the number of atoms per molecule in the QM9 molecular dataset, compared to the empirical distribution of the number of atoms in molecules generated by molecular design computation models trained on the QM9 molecular dataset, in several exemplary embodiments.

[0068] [Figure 11A]A graph showing the cumulative distribution function (CDF) of strain energy for molecules in the Molecular Geometry Ensemble (GEOM) drug dataset, molecules generated by conventional generative models, and molecules generated by molecular design computation models trained on the GEOM drug dataset, in several exemplary embodiments.

[0069] [Figure 11B] A graph showing the empirical distribution of the number of atoms per molecule in the Molecular Geometry Ensemble (GEOM) drug dataset, compared to the empirical distribution of the number of atoms in molecules generated by molecular design computation models trained on the GEOM drug dataset, in several exemplary embodiments.

[0070] [Figure 12A] A schematic diagram showing a comparison of seed generation performance for molecular geometric ensemble (GEOM) drugs in discrete voxelization space and latent voxelization space, according to several exemplary embodiments.

[0071] [Figure 12B] This is a schematic diagram showing a comparison of seed generation performance for PubChem drugs in discrete voxelization spaces and latent voxelization spaces, according to several exemplary embodiments.

[0072] [Figure 12C] Molecular graph diagrams of additional examples of molecules generated by performing seed generation in the latent voxelization space for actual drugs.

[0073] [Figure 12D] Molecular graph diagrams of additional examples of molecules generated by performing de novo synthesis in the latent voxelization space on actual drugs.

[0074] [Figure 13] A graph showing a comparison of seed generation performance for molecular geometric ensemble (GEOM) drugs in discrete voxelization space and latent voxelization space, according to several exemplary embodiments.

[0075] [Figure 14] A block diagram illustrating an example of a computing system in several exemplary embodiments.

[0076] In practical terms, similar reference numbers indicate similar structures, features, or elements. [Modes for carrying out the invention]

[0077] Detailed explanation Generating new molecules with desired properties is a crucial task in chemistry, with applications spanning many scientific fields. In the context of drug discovery, conventional computational techniques for generating molecules with drug-like properties require searching the molecular space (or chemical space) occupied by all possible chemical compounds (e.g., all possible combinations of atoms of two or more chemical elements). For example, some search-based methods may involve scoring and ranking different molecules within the molecular space based on one or more drug-like properties such as affinity, specificity, biological activity, and development suitability. However, The aforementioned molecular space, estimated to contain the possible chemical compounds of TIFF2026518659000001.tif5170, is extraordinarily large and scales exponentially with molecular size (e.g., the number of constituent atoms). Even a tiny fraction of this molecular space could contain billions to trillions of molecules. With state-of-the-art computing resources, conventional search-based methods can only explore small portions of the molecular space, such as small regions of the molecular space selected based on prior domain knowledge. This limitation in search scope means that conventional search-based methods are likely to miss molecules with more optimal properties. Moreover, conventional search-based methods do not explore the molecular space in a principled way, preventing the generation process from being conditioned to specific properties.

[0078] In addition, whether a molecule exhibits specific desired properties may depend on its conformation (or three-dimensional structure). For example, the binding affinity between a drug molecule and a target molecule (e.g., a protein, nucleic acid, etc.) may depend on the drug molecule's ability to adopt a conformation (or three-dimensional structure) complementary to the target molecule's conformation. Furthermore, molecules are flexible, meaning that a single molecule can adopt one of many possible conformations (or three-dimensional structures). In some cases, a group of the same molecules can exist as a collection of many different conformations in equilibrium with each other, but not all possible conformations are associated with the desired properties. In relation to binding affinity, for example, the biologically active conformation of a molecule may be one or more of the conformations exhibited by the molecule in solution, or new conformations induced by interaction with the target molecule. However, one-dimensional representations of molecules (e.g., Simplified Molecular Input Line Notation (SMILES) strings) or two-dimensional representations (e.g., molecular graphs) do not adequately capture the conformation (or three-dimensional structure) of a molecule. Therefore, when a molecular design calculation model operates on a one-dimensional or two-dimensional representation of the input molecule, the resulting output molecule may not be able to exhibit a conformation (or three-dimensional structure) associated with one or more desired properties.

[0079] Various exemplary embodiments of this disclosure can improve current state-of-the-art computational resources by providing molecular design computation models that can generate output molecules by exploring molecular space (or chemical space) in principle, rather than performing an indiscriminate search of a limited portion of molecular space. For example, in some cases, the molecular design computation model may be trained to approximate a data distribution of molecules that exhibit one or more desired properties (e.g., drug-like properties such as affinity, specificity, biological activity, development suitability, etc.). Training the molecular design computation model may involve determining the parameters of a function (e.g., a score function) such that the output of the function is a value that indicates a change in density across the data distribution. In some cases, the molecular design computation model may sample the data distribution to generate output molecules that also exhibit one or more desired properties. For example, in some cases, the molecular design computation model may sample the data distribution by denoising input molecules, such as voxelized representations of the input molecules, over multiple sampling iterations. During each sampling iteration, the molecular design computation model may update the input molecules to remove some of the noise present in the input molecules. By doing so, updated molecules (e.g., voxelized representations of the updated molecules) that constitute the samples selected from the data distribution can be generated. As described in more detail below, sampling can be guided by a function such that each consecutive sample (or updated molecule) is selected from a region of the data distribution with an increasingly higher density, where it is more likely to be occupied by molecules exhibiting one or more properties.

[0080] In some exemplary embodiments, the likelihood that the output molecule exhibits one or more desired properties can be increased (or maximized) by a molecular design computation model operating on a three-dimensional representation of the input molecule. For example, in some cases, the molecular design computation model may generate the output molecule by denoising the three-dimensional representation of the input molecule over at least, for example, multiple sampling iterations. In some cases, the molecular design computation model may generate the output molecule by denoising a voxelized representation of the input molecule instead of a conventional three-dimensional representation of the input molecule. A conventional three-dimensional representation of an input molecule, such as a point cloud representation of the input molecule, can specify the conformation (or three-dimensional structure) of the input molecule by specifying at least the coordinates of the constituent atoms (e.g., in Euclidean space). However, a conventional three-dimensional representation of an input molecule may impose some limitations on the generation process. For example, for a molecular design computation model to operate on a conventional three-dimensional representation of the input molecule, the number of atoms in the output molecule generated from it must be known in advance. Denoising conventional three-dimensional representations of input molecules may also require specific workarounds to allow molecular design computation models to approximate the distribution of atom types in the output molecule, which forms a dispersed distribution, while the positions of atoms in the output molecule (e.g., atomic coordinates in Euclidean space) form a continuous distribution. Furthermore, conventional three-dimensional representations of input molecules may not adequately capture long-range dependencies that exist across multiple atoms, especially as the number of constituent atoms increases.

[0081] In some exemplary embodiments, a voxelized representation of a molecule (e.g., an input molecule) can overcome the aforementioned limitations by representing the input molecule as a continuous distribution of atomic density across a voxel grid, centered on the atomic coordinates of the individual atoms present in the molecule. For example, in a graph network representation of a molecule, the dependency between two adjacent atoms may be represented by interconnection edges. However, these edges may not adequately capture longer-range dependencies, such as dependencies between non-adjacent atoms. In contrast, even when the input molecule contains a large number of atoms, a voxelized representation of a molecule can better capture long-range dependencies between distant atoms. Furthermore, a molecular design computation model can operate on a voxelized representation of an input molecule to generate an output molecule without prior knowledge of the number of atoms present in the output molecule. This is because the molecular design computation model can freely add or remove different types of atoms by updating the distribution of atomic density across the voxel grid. The voxelized representation of the input molecule also represents both the type and position of atoms within the input molecule, thereby eliminating the need for workarounds to reconcile two different types of data distributions (e.g., a discrete distribution of atom types and a continuous distribution of atom positions).

[0082] In some exemplary embodiments, a voxelized molecular representation of a molecule, such as an input molecule, can represent each atom in the molecule (e.g., the input molecule) as a continuous (e.g., Gaussian) density across one or more voxels in a voxel grid. In this context, the voxel grid is a three-dimensional grid of voxels organized into continuous layers of rows and columns. Various examples of voxel grids described herein may include multiple voxels, each of which is a volume element (e.g., a three-dimensional cube) at the intersection of rows and columns. Each volume element may have a predetermined size that is the same for all voxels in the voxel grid, or it may not be the same for all voxels. When the input molecule is a drug molecule, the voxelized representation of the input molecule is: The voxel grid may include TIFF2026518659000002.tif5170. In some cases, each voxel in the voxel grid may be associated with a value indicating the atomic density at its corresponding location. For example, a first voxel associated with a higher atomic density may be more likely to be part of an atom than a second voxel associated with a lower atomic density. It should be understood that the volume of an individual atom may span one or more voxels. In some cases, the atomic density may be centered on an atom present in the input molecule, meaning that the atomic density of an individual atom spanning multiple voxels may be centered on the voxel containing the center of that atom. A voxel with an atomic density of 0 may be far from any atom in the input molecule, while a voxel with an atomic density of 1 may be at the center of an atom in the input molecule. Furthermore, in some cases, the voxelized representation of a molecule (e.g., an input molecule) may include multiple channels, each corresponding to a type of atom that may be present in the input molecule. The “atomic type” may refer to the individual chemical element corresponding to that atom. The voxelized representation may include multiple channels, one for each type of atom present in the molecule, or at least one for each type of heavy atom. For example, in some cases, the voxelized representation of a molecule (e.g., an input molecule) may include a first channel corresponding to a first atomic type that may be present in the input molecule (e.g., carbon (C) atoms) and a second channel corresponding to a second atomic type that may be present in the input molecule (e.g., nitrogen (N) atoms). Each voxel in the first channel may be associated with a value indicating the density of atoms of the first atomic type at the corresponding position, while each voxel in the second channel may be associated with a value indicating the density of atoms of the second atomic type at the corresponding position. Thus, as will be described in more detail below, the molecular design calculation model can denoise the voxelized representation of the input molecule by updating the atomic density of one or more voxels in at least one channel of the voxelized representation of the input molecule at least, for example, over multiple sampling iterations.In other words, in some cases, the term “denoising” refers to updating the voxelized representation of an input molecule, which may include updating the atomic density of at least one voxel in the voxelized representation of the input molecule. In some cases, updating the atomic density of a voxel within one channel of the voxelized representation of the input molecule may change the likelihood that the voxel is part of the type of atom associated with that channel.

[0083] In some exemplary embodiments, the molecular design computation model may denoise the input molecules (e.g., voxelized representations of the input molecules) over multiple sampling iterations, with each sampling iteration generating an updated voxelized representation different from the original input molecule's voxelized representation. In some cases, each updated voxelized representation may include a sample selected from a data distribution of molecules (e.g., voxelized representations of molecules) exhibiting one or more desired properties. In this context, the term “data distribution” can refer to a collection of molecules with different molecular compositions and conformations (or three-dimensional structures). Molecules exhibiting one or more desired properties may cluster in higher density regions of the data distribution, meaning the molecular design computation model needs to sample each updated voxelized representation from higher density regions of the data distribution. However, this data distribution may be too high-dimensional to be directly approximated. For example, a normalization constant is needed to compute a probability density function (PDF) characterizing the probabilities of different molecules in the data distribution. In the case of molecular design, this normalization constant may correspond to the total number of molecules in the data distribution, which may be impossible to estimate. Therefore, in some cases, molecular design computation models can be trained to approximate the data distribution by determining a function, such as a score function, that estimates the gradient (or change in density) across the data distribution. As will be described in more detail below, molecular design computation models can use a function to induce sampling of updated voxelized representations from the data distribution such that each successive sample is selected from a region of the data distribution with progressively higher density.

[0084] As described above, denoising an input molecule over multiple sampling iterations, including sequential updates to the voxelized representation of the input molecule, may in some cases be equivalent to selecting a series of samples from a data distribution of the molecule (e.g., a data distribution of the voxelized representation of the molecule) where each sample corresponds to an updated voxelized representation different from the voxelized representation of the input molecule. For example, in some cases, the voxelized representation of the input molecule may be denoised by updating the atomic density of at least one or more voxels in at least one channel of the voxel grid forming the voxelized representation of the input molecule. In some cases, molecules in a higher density region of the data distribution may exhibit one or more desired properties, including drug-like properties such as affinity, specificity, biological activity, and suitability for development. Therefore, when operating on a three-dimensional representation of an input molecule, such as a voxelized representation of the input molecule, the likelihood that the resulting conformation (or three-dimensional structure) of the output molecule, selected from a higher density region of the data distribution, will exhibit one or more desired properties may increase.

[0085] In some cases, a molecular design computation model may be trained to approximate a data distribution by training the molecular design computation model using a training dataset of known molecules exhibiting one or more desired properties (e.g., the PubChem dataset, the QM9 molecular dataset, the Molecular Geometry Ensemble (GEOM) drug dataset). For example, in some cases, a molecular design computation model may be trained to approximate a data distribution by determining, at least, a function (e.g., a score function) that approximates different densities across the data distribution by Bayesian estimation. In some cases, the function may be parameterized by the molecular design computation model, meaning that the parameters of the function (e.g., the score function) are parameters of the molecular design computation model that have been tuned when the molecular design computation model is trained to approximate a data distribution. In some cases, high-density regions of the data distribution may be occupied by molecules similar to one or more known molecules exhibiting the desired properties, while low-density regions of the data distribution may be occupied by molecules not similar to one or more known molecules exhibiting the desired properties. For example, a score function for a data distribution may represent transitions between different density regions of the data distribution, such as transitions between higher and lower density regions of the data distribution. Thus, once trained, a molecular design computation model may sample the data distribution based on the score function so that each consecutive sample (or molecule) is selected from an increasingly higher density region of the data distribution.

[0086] In some exemplary embodiments, a molecular design computation model may be trained to denoise corrupted three-dimensional representations of known molecules from a training dataset and reconstruct the original three-dimensional representations of the known molecules. For example, in some cases, the corrupted three-dimensional representations of known molecules may be generated by corrupting the three-dimensional representations of known molecules with noise (Gaussian noise, such as isotropic Gaussian noise). Training the molecular design computation model may involve tuning one or more parameters of the molecular design computation model (e.g., weights, biases, etc.) to reduce (or minimize) the difference (e.g., mean squared error (MSE)) between the reconstructed three-dimensional representations of known molecules and the original three-dimensional representations of known molecules.

[0087] In some exemplary embodiments, to avoid overfitting the molecular design model to known molecules in the training dataset, the molecular design model may be trained to reconstruct a noisy version of the three-dimensional representation of the known molecules in the training dataset instead of the original three-dimensional representation. That is, the three-dimensional representation of each known molecule in the training dataset may be contaminated with additional noise, but this noise should not be confused with the noise that the molecular design model is trained to remove from the corrupted three-dimensional representation of each known molecule in the training dataset. In other words, in some cases, the molecular design model may be trained on a training dataset containing noisy three-dimensional representations of known molecules and their corrupted versions. As will be described in more detail below, in some cases, the noisy three-dimensional representation of a known molecule may be generated by contaminated the three-dimensional representation of the known molecule (e.g., a voxelized representation) with a first amount of noise (e.g., Gaussian noise such as isotropic Gaussian noise) to smooth the density of the data distribution of the known molecule while still retaining at least a portion of the conformation (e.g., three-dimensional structure) of the known molecule, thereby obtaining a noisy representation of the known molecule. Subsequently, the noisy three-dimensional representation of the known molecule may be further corrupted with a second amount of noise (e.g., Gaussian noise such as isotropic Gaussian noise) to generate a corrupted three-dimensional representation. In some cases, the molecular design computation model may be trained to denoise the corrupted three-dimensional representation of the known molecule by, for example, removing the second amount of noise, and to reconstruct the noisy three-dimensional representation of the known molecule (still containing the first amount of noise).Furthermore, in some cases, training of the molecular design computation model may include gradient-based Markov chain Monte Carlo (MCMC) sampling (e.g., Langevin Markov chain Monte Carlo (MCMC) sampling), and the parameters of the molecular design computation model are adjusted over successive sampling iterations to increase the similarity (e.g., reduce the mean squared error (MSE)) between the three-dimensional representation of each known molecule reconstructed by the molecular design computation model from the corresponding corrupted three-dimensional representations of known molecules in the training dataset and the noisy three-dimensional representation of the sample molecules in the training dataset. The score function thus derived may capture data distributions with smoother density transitions, thereby mitigating the phenomenon of mode collapse when the molecular design computation model is not very robust and can only generate output molecules of a limited selection (e.g., those in the immediate vicinity of known molecules in the data distribution).

[0088] As will be described in more detail below, during estimation, a trained molecular design computation model may be applied to generate one or more output molecules by denoising the three-dimensional representation of an input molecule. In some cases, the input molecule may be a random molecule (e.g., a molecule in which the type and / or position of atoms are randomly selected) or a known molecule having one or more undesirable properties, meaning that the three-dimensional representation of the input molecule may contain at least some noise that needs to be removed so that the three-dimensional representation of the output molecule generated therefrom matches one or more desired properties. The molecular design computation model may do this, for example, by traversing the smoothing density of the noisy data distribution of the noisy three-dimensional representation of the molecule through one or more iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling (e.g., Langevin Markov chain Monte Carlo) toward increasingly higher density regions of the data distribution. Each iteration of gradient-based Markov chain Monte Carlo (MCMC) sampling may include updating the three-dimensional representation of the input molecule, which is equivalent to selecting the noisy three-dimensional representation of one or more molecules from different locations within a noisy data distribution. Even when selected from a noisy data distribution, the molecules corresponding to these noisy three-dimensional representations may be less distorted than the original three-dimensional representation of the input molecule. Such molecules corresponding to these noisy three-dimensional representations may better match the input molecule with one or more desired properties. In some cases, the noisy three-dimensional representations of molecules selected from a noisy data distribution may undergo further denoising to reconstruct the corresponding molecule by mapping the noisy three-dimensional representation of each molecule from the noisy data distribution to the corresponding clean three-dimensional representation of the molecule in the true data distribution of the one or more desired properties. It should be understood that sampling from a noisy data distribution may offer more advantages than sampling from the true data distribution of the one or more desired properties.For example, sampling from a noisy data distribution of molecules exhibiting one or more desired properties with smoother density transitions may be less susceptible to mode collapse than sampling from the true data distribution. In some cases, this may be because the noisy data distribution has fewer steep or abrupt gradients than the true data distribution, where steep gradients restrict sampling to the immediate vicinity of known molecules characterizing the true data distribution. In other words, molecular design computation models may produce outputs with limited diversity when sampling from the true data distribution (e.g., the aforementioned phenomenon called "mode collapse"), but sampling from a noisy distribution may increase the diversity of the model's output. Moreover, sampling from a noisy data distribution occupied by a noisy voxelized representation of a molecule may offer additional advantages over sampling from a noisy data distribution occupied by a noisy conventional three-dimensional representation of a molecule, such as a point cloud representation of a molecule. For example, in some cases, molecular design computation models may operate on voxelized molecular representations and be trained to generate large drug-like molecules with greater ease, plausibility, expressiveness, and scalability. Unlike when operating on conventional three-dimensional molecular representations (e.g., point cloud representations), operating on voxelized molecular representations may allow the disclosed molecular design computation models to function without requiring a specification of the number of atoms present in the output molecule and without workarounds to reconcile the discrete distribution of atom types associated with each molecule with the continuous distribution of atomic positions.

[0089] Despite the aforementioned advantages, operating on voxelized molecular representations can impose a considerable computational burden, potentially scaling exponentially with molecular size (e.g., the number of constituent atoms). For example, even a small molecule containing 10 heavy atoms already requires a [32×32×32] voxel grid with 32,000 features (or atomic density values) per molecule, but larger, more realistic drug-like molecules may require at least twice that size of voxel grid, with exponentially more points (e.g., a [64×64×64] voxel grid with 260,000 features (or atomic density values) per molecule). In some cases, applying molecular design computation models to operate on voxelized representations of larger, more realistic drug-like molecules can be a challenging task, as is the case when training molecular design computation models on large training datasets (e.g., training datasets containing millions of voxelized representations of known molecules) to learn a wider variety of molecular spaces (or chemical spaces). Furthermore, even when candidate molecules are realistic and reasonable, a large portion of the candidate molecules generated by molecular design computation models may not be successfully synthesized in the laboratory. Therefore, it may be desirable to apply molecular design computation models to generate tens of thousands or even millions of candidate molecules. The computational load associated with generating molecules, particularly larger or more numerous molecules, can be reduced by molecular design computation models that operate on lower-dimensional embedding representations of voxelized molecular representations. For example, in some cases, a molecular design computation model may be trained to generate output molecules by denoising the embedding representation of a three-dimensional representation of an input molecule. As will be described in more detail below, in some cases, a molecular design computation model may be trained on a training dataset containing corrupted embedding representations of one or more sample molecules exhibiting desired properties, each generated by encoding noisy three-dimensional representations (e.g., voxelized representations) of one or more known molecules exhibiting desired properties before the resulting embedding representation is corrupted by the addition of noise (e.g., isotropic Gaussian noise).

[0090] In some exemplary embodiments, by encoding a three-dimensional representation of a molecule, such as a voxelized representation of a molecule, the three-dimensional representation of a molecule (e.g., a voxelized molecular representation) can be projected from a higher-dimensional discrete space occupied by the three-dimensional representation of the molecule (e.g., a discrete voxelized space occupied by the voxelized molecular representation) to a lower-dimensional representation in a lower-dimensional latent space occupied by the corresponding molecular embedding representation. In other words, encoding a three-dimensional representation of a molecule, such as a voxelized representation of a molecule, can take a three-dimensional representation of the molecule in an occupied higher-dimensional discrete space (e.g., a voxelized molecular representation) as input and produce a lower-dimensional representation of the molecule in a lower-dimensional latent space (i.e., a molecular embedding representation corresponding to the input three-dimensional representation) as output. Encoding a three-dimensional representation of a molecule, such as a voxelized representation of a molecule, can be performed using a machine learning model trained to identify the latent space representation of the input molecule's three-dimensional representation from which the input molecule's three-dimensional representation can be reconstructed. In some cases, each embedding representation in latent space may be a latent space representation of the corresponding voxelized representation of the molecule. Therefore, the embedding representation of the voxelized representation of the molecule may have different dimensions or features than the voxelized representation of the molecule. For example, in some cases, encoding the voxelized representation of the molecule may reduce the dimensions of the voxelized representation or the features present within it. Therefore, the computational load of denoising the voxelized representation of the molecule and generating one or more output molecules from it may be reduced by a molecular design computation model that operates on the embedding representation of the voxelized representation of the molecule rather than directly on the voxelized representation of the molecule, because the embedding representation contains at least a smaller amount of features. Furthermore, in some cases, the molecular design computation model may be trained to approximate a noisy data distribution of one or more molecules exhibiting one or more desired characteristics, from which one or more output molecules can be generated by sampling. This noisy data distribution, which may be occupied by the noisy embedding representation of the voxelized representation of the molecule, may exhibit smoother density transitions than the corresponding true data distribution.Thus, noisy data distributions may support more efficient sampling (e.g., via gradient-based Markov chain Monte Carlo (MCMC) sampling, such as Langevin Markov chain Monte Carlo) because they may exhibit fewer steep gradient changes that would otherwise hinder molecular design computation models from properly exploring the data distribution when sampling from them.

[0091] Figures 1A and 1B depict system diagrams showing various examples of molecular design system 100 according to several exemplary embodiments. Referring to Figures 1A and 1B, the molecular design system 100 may, in some cases, include a molecular design engine 110, a training engine 120, and a client device 130. In the examples of molecular design system 100 shown in Figures 1A and 1B, the molecular design engine 110, the training engine 120, and the client device 130 may be communicably coupled via a network 140. The client device 130 may be a processor-based device including, for example, a workstation, desktop computer, laptop computer, smartphone, tablet computer, or wearable device. The network 140 may be a wired network and / or wireless network including, for example, a local area network (LAN), a virtual local area network (VLAN), a wide area network (WAN), a public land mobile network (PLMN), or the internet. In the example shown in Figures 1A and 1B, the molecular design calculation model 115 may include a denoising model 117 trained to produce an output molecule 162 by denoising at least the input molecule 152. The denoising model is a machine learning model trained to take in a corrupted 3D representation of a molecule or a lower-dimensional embedding representation of a molecule as input, where the corrupted 3D representation of a molecule is a 3D representation of the molecule with noise added, and to produce a corresponding denoised 3D representation of a molecule or a lower-dimensional embedding representation of a molecule as output, using training data that includes 3D representations of a plurality of known molecules and their corresponding corrupted 3D representations. The known molecules in the training data may include a plurality of molecules that exhibit one or more desired properties. The denoising model may be an artificial neural network. The denoising model may be a deep learning model. The denoising model may be an encoder-decoder 3D convolutional neural network (CNN).For example, in some cases, the molecular design calculation model 115 may apply a denoising model 117 to denoise the three-dimensional representation of the input molecule 152 in order to generate the output molecule 162, or alternatively, denoise the embedding representation 154 of the three-dimensional representation of the input molecule 152. In some cases, the denoising model 117 may denoise the three-dimensional representation (or its embedding representation 154) of the input molecule 152 over multiple consecutive sampling iterations, so that a portion of the noise present in the three-dimensional representation (or its embedding representation 154) of the input molecule 152 is removed with each sampling iteration.

[0092] As will be explained in more detail below, denoising the three-dimensional representation of the input molecule 152 may alter the composition and / or conformation (or three-dimensional structure) of the input molecule 152 so that the composition and conformation (or three-dimensional structure) of the resulting output molecule 162 matches the composition and conformation (or three-dimensional structure) of a molecule exhibiting one or more desired properties. If the output molecule 162 is a drug molecule, for example, one or more desired properties may include drug-like properties such as affinity, specificity, biological activity, and suitability for development. In some cases, whether the output molecule 162 exhibits a particular desired property may be conditional on the output molecule 162 exhibiting the corresponding conformation (or three-dimensional structure). Therefore, in some cases, a molecular design calculation model 115 that applies a denoising model 117 to operate on a three-dimensional representation (or its embedded representation 154) of the input molecule 152 instead of a one-dimensional or two-dimensional representation of the input molecule 152 increases the likelihood that the resulting output molecule 162 exhibits a conformation (or three-dimensional structure) that matches one or more desired properties.

[0093] In some exemplary embodiments, the molecular design computation model 115, including a denoising model 117, may be trained to learn or approximate a data distribution of molecules exhibiting one or more desired properties (e.g., drug-like properties such as affinity, specificity, biological activity, and suitability for development). For example, in some cases, the molecular design computation model 115 may be trained to approximate a data distribution of molecules exhibiting one or more desired properties based on a training dataset of known molecules exhibiting one or more desired properties (e.g., the PubChem dataset, the QM9 molecular dataset, the Molecular Geometry Ensemble (GEOM) drug dataset). As will be described in more detail below, the molecular design computation model 115 may be trained to approximate a noisy data distribution occupied by noisy three-dimensional representations of known molecules exhibiting one or more desired properties. Furthermore, the molecular design calculation model 115 may be trained to operate in either a discrete space occupied by three-dimensional representations (e.g., voxelized representations) of molecules exhibiting one or more desired properties, or alternatively, a latent space occupied by embedding representations of three-dimensional representations of molecules (e.g., embedding representations of voxelized representations), which are lower-dimensional representations of molecules.

[0094] To further illustrate, Figure 1A depicts an example of the molecular design engine 110, where the molecular design calculation model 115 is trained to operate in discrete space (three-dimensional voxelized representation), while the molecular design calculation model 115 included in the example of the molecular design engine 110 shown in Figure 1B may be trained to operate in latent space. Latent space may contain continuous values ​​(embedding representations) of multiple features. In any example of the molecular design engine 110, the molecular design calculation model 115 may be trained to approximate a noisy data distribution by being trained, for example, on a noisy three-dimensional representation of a molecule exhibiting one or more desired properties. For example, the example of the molecular design calculation model 115 shown in Figure 1A may be trained to approximate a noisy discrete distribution occupied by a noisy three-dimensional representation of a molecule, while the example of the molecular design calculation model 115 shown in Figure 1B may be trained to approximate a noisy latent distribution occupied by a noisy embedding representation of a three-dimensional representation of a molecule.

[0095] Referring first to Figure 1A, the molecular design engine 110 may, in some cases, include a molecular design calculation model 115 and a reconstruction model 118. Alternatively, Figure 1B depicts another example of the molecular design engine 110, which may also include an encoder 111 and a decoder 119 in addition to the molecular design calculation model 115 and the reconstruction model 118. In some exemplary embodiments, the molecular design engine 110 may apply the molecular design calculation model 115 to generate an output molecule 162 based on at least an input molecule 152. For example, in the example of the molecular design engine 110 shown in Figure 1A, the molecular design calculation model 115 may operate on a three-dimensional representation of the input molecule 152, which may optionally be a voxelized representation of the input molecule 152. In doing so, the molecular design calculation model 115 may operate in a discrete voxelized space occupied by noisy voxelized representations of different molecules (e.g., molecules exhibiting one or more desired properties). Alternatively, in the example of the molecular design engine 110 shown in Figure 1B, the molecular design calculation model 115 may operate on an embedding representation 154 of the input molecule 152. As will be described in more detail below, the embedding representation 154 of the input molecule 152 may be generated by an encoder 111 that encodes a three-dimensional representation (e.g., a voxelized representation) of the input molecule 152. In this modification of the molecular design engine 110, the molecular design calculation model 115 may operate in a latent space occupied by noisy embedding representations of three-dimensional representations (e.g., voxelized representations) of different molecules.

[0096] In some exemplary embodiments, the molecular design calculation model 115 may include a denoising model 117 trained to denoise a three-dimensional representation of an input molecule 152 based on a function 175, so that the resulting three-dimensional representation of the output molecule 162 is sampled from a denser region of the data distribution of one or more molecules exhibiting desired properties. Figure 1A depicts an example of a denoising model 117 trained to denoise a three-dimensional representation of an input molecule 152, while Figure 1B depicts another example of a denoising model 117 trained to denoise an embedding representation 154 of the three-dimensional representation of the input molecule 152. In some cases, the denoising model 117 may denoise the three-dimensional representation of the input molecule 152 (e.g., a voxelized representation of the input molecule 152), or alternatively, its embedding representation 154, over multiple time steps. In some cases, the denoising performed at each time step may be equivalent to selecting one or more samples (e.g., intermediate molecules) from different locations in the data distribution. In some cases, function 175 may be a scoring function that outputs a value (e.g., a score) indicating a local density change at a specific location in the data distribution (e.g., the location occupied by a particular molecule). Thus, denoising of the input molecules 152 may be performed based on the output of the scoring function so that each successive sample (or molecule) is selected from an increasingly dense region of the data distribution.

[0097] In some exemplary embodiments, denoising of the input molecule 152 may include updating a three-dimensional representation (e.g., a voxelized representation) (or its embedding representation 154) of the input molecule 152 that can represent the composition and conformation (or three-dimensional structure) of the input molecule 152, in order to increase the likelihood that the resulting output molecule 162 exhibits one or more desired properties in the data distribution of molecules. Optionally, the molecular design calculation model 115 may apply a denoising model 117 to modify the three-dimensional representation of the input molecule 152 (or its embedding representation 154) over one or more iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling (e.g., Markov chain Monte Carlo (MCMC) with Langevin dynamics). For example, optionally, each iteration of gradient-based Markov chain Monte Carlo (MCMC) sampling may include selecting a sample (or molecule) from the first data distribution that includes one or more modifications to the three-dimensional representation (or its embedding representation 154) of the input molecule 152. As described above, in some cases, sampling from the data distribution may be induced by function 175 (e.g., a score function). For example, if function 175 is a score function, function 175 may output a value (e.g., a score) for each sample (or numerator) selected from the data distribution that corresponds to the change in density observed at the location in the data distribution occupied by the sample (or numerator). Thus, in some cases, sampling from the data distribution may be induced by function 175, and as a result, each consecutive sample is selected from a region of the data distribution with progressively higher density.

[0098] To illustrate further, in some cases, the denoising model 117 may be applied to update the three-dimensional representation (e.g., voxelized representation) (or its embedding representation 154) of the input molecule 152 by, for example, selecting a first sample and a second sample from the data distribution. It should be understood that each of the first and second samples may correspond to the modified three-dimensional representation of the input molecule 152 in the example shown in Figure 1A, or, in the case of Figure 1B, to the modified embedding representation of the three-dimensional representation of the input molecule 152. If function 175 is a score function, function 175 may assign a first value (e.g., a first score) to the first sample to indicate a more positive local change (e.g., an increase or a smaller decrease) in the density of the data distribution at a first position of the first sample, and assign a second value to the second sample to indicate a less positive local change (e.g., a smaller increase or decrease) in the density of the data distribution at a second position of the second sample. In some cases, the molecular design calculation model 115 may apply a denoising model 117 to select a third sample (e.g., another modified three-dimensional representation or another modified embedding representation) from the data distribution by further modifying the first sample to sample the third sample from a region of the data distribution with a higher density than the first and second samples.

[0099] In some exemplary embodiments, the molecular design calculation model 115 may apply a denoising model 117 to denoise the voxelized representation (or its embedding representation 154) of the input molecule 152, instead of a conventional three-dimensional representation of the input molecule 152, such as a point cloud representation of the input molecule 152. For example, in some cases, the voxelized representation of the input molecule 152 may represent the types and locations of atoms present in the input molecule 152 as a continuous (e.g., Gaussian) density across a three-dimensional voxel grid. To indicate the locations of atoms present in the input molecule 152, each voxel in the voxel grid may be associated with a value indicating the atomic density at the corresponding location. In some cases, the atomic density associated with a particular voxel in the voxel grid may correspond to the likelihood that that voxel is part of the atoms at that location. For example, a first voxel with a higher atomic density may be more likely to be part of the atoms forming the input molecule 152 than a second voxel with a lower atomic density. Therefore, the voxelized representation of the input molecule 152 may represent the positions of the atoms in the input molecule 152 by distinguishing between voxels in the voxel grid that form a portion of the atoms in the input molecule 152 and voxels in the voxel grid that do not form a portion of the atoms in the input molecule 152, based on the atomic density associated with each voxel in the voxel grid. In some cases, the atoms forming the input molecule 152 may be located at the positions of voxels associated with atomic densities that satisfy one or more thresholds.

[0100] In some exemplary embodiments, the voxelized representation of the input molecule 152 may include one or more channels, each corresponding to a type of atom that may be present in the input molecule 152. For example, in some cases, the voxelized representation of the input molecule 152 may include a separate channel for each type of heavy atom that may be present in the input molecule 152. By including separate channels for different types of atoms in the voxelized representation of the input molecule 152, the discrete distribution typically associated with atomic types seen in conventional three-dimensional representations (e.g., point cloud representations) can be avoided. Instead, the voxelized representation of the input molecule 152 may represent the types and locations of atoms in the input molecule 152 as one or more continuous (e.g., Gaussian) densities across the three-dimensional voxel grid described above. For example, the voxelized representation of the input molecule 152 may include a first channel representing a first type of atom that may be present in the input molecule 152 (e.g., carbon (C) atoms). The presence of a first type of atom in the input molecule 152 and the respective positions thereof can be represented by a first continuous (e.g., Gaussian) density across a first channel in the voxelized representation of the input molecule 152. Note that the density is continuous in the sense that the values ​​associated with each voxel can take continuous values ​​(e.g., values ​​within a continuous, bounded, or unbounded distribution). In some cases, the voxelized representation of the input molecule 152 may further include a second channel representing a second type of atom that may be present in the input molecule 152 (e.g., nitrogen (N) atoms). The presence of the second type of atom and the respective positions thereof can be represented by a second continuous (e.g., Gaussian) density across the second channel in the voxelized representation of the input molecule.

[0101] Unlike conventional three-dimensional representations of input molecule 152 (e.g., point cloud representations) that represent the types and positions of atoms in input molecule 152 as two distinct types of distributions (e.g., discrete distributions of atom types and continuous distributions of atom positions), the voxelized representation of input molecule 152 can represent the types and positions of atoms in input molecule 152 together as one or more continuous (e.g., Gaussian) distributions in the manner described above. Thus, the molecular design calculation model 115 can apply the denoising model 117 to work with the voxelized representation of input molecule 152 without workarounds to harmonize the two distinct types of distributions required in the conventional three-dimensional representation of input molecule 152. The voxelized representation of input molecule 152 can also represent the conformation (or three-dimensional structure) of input molecule 152 better than the conventional three-dimensional representation of input molecule 152. For example, the voxelized representation of input molecule 152 can capture long-range dependencies between distant atoms, even when input molecule 152 contains a large number of atoms. Furthermore, the molecular design calculation model 115 may apply a denoising model 117 to denoise the voxelized representation of the input molecule 152 and generate the output molecule 162 without any prior knowledge of the amount of molecules present in the output molecule 162.

[0102] In some exemplary embodiments, the training engine 120 may be trained to generate one or more training samples to be included in the training dataset. Figure 1A depicts an example of the training engine 120 in which the corrupted engine 121 generates each training sample by adding noise (e.g., Gaussian noise such as isotropic Gaussian noise) to a noisy three-dimensional representation 182 of the sample molecule (e.g., a known molecule exhibiting one or more desired properties), thereby generating a corrupted three-dimensional representation 184 of the sample molecule. It should be understood that the corrupted engine 121 may add noise to the already noisy three-dimensional representation 182 of the sample molecule in order for the molecular design calculation model 115 to be trained to approximate a noisy data distribution of one or more molecules exhibiting one or more desired properties that has smoother density transitions than the true data distribution of molecules exhibiting one or more properties. Therefore, in some cases, the molecular design calculation model 115, including the denoising model 117, may be trained to denoise the corrupted three-dimensional representation of the sample molecule and, for each training sample, to reconstruct the corresponding noisy three-dimensional representation of the sample molecule, rather than the clean (or original) three-dimensional representation of the sample molecule. Doing so may be equivalent to sampling from a noisy data distribution of molecules that exhibit one or more desired properties, rather than the true distribution of molecules.

[0103] Alternatively, Figure 1B depicts another example of the training engine 120 in which the encoder 111 first encodes the noisy three-dimensional representation 182 of the sample molecule to generate its embedding representation 186 before the corruption engine 121 adds noise to generate a corrupted embedding representation 188 of the noisy three-dimensional representation 182 of the sample molecule. In this modification of the molecular design engine 110, the molecular design computation model 115 may be trained to approximate a noisy latent distribution of noisy embedding representations of three-dimensional representations of molecules exhibiting one or more desired properties, instead of the noisy but discrete distribution that the molecular design computation model 115 is trained to approximate in the example shown in Figure 1A. To achieve this result, the training engine 120 may generate each training sample in the training dataset to include the corrupted embedding representation 188. For example, Figure 1B shows that the training engine may further include an encoder 111, which may first encode a noisy three-dimensional representation 182 of a sample molecule (e.g., a known molecule exhibiting one or more desired properties) to generate an embedding representation 186 before the corrupted engine 121 adds noise to it to generate a corrupted embedding representation 188. In some cases, a molecular design calculation model 115 including a denoising model 117 may be trained to denoise the corrupted embedding representation 188 and reconstruct the embedding representation 186 from it. The denoising model 117, the encoder 111, and the corresponding decoder may be trained together.

[0104] In some exemplary embodiments, training the molecular design calculation model 115 may include tuning one or more parameters (e.g., weights, biases, etc.) of the denoising model 117. In the example shown in Figure 1A, the parameters of the denoising model 117 may be tuned so that the denoising model 117 can reconstruct the corresponding noisy three-dimensional representation 182 of the sample molecule from the corrupted three-dimensional representation 184 of the sample molecule. For example, as will be described in more detail below, one or more parameters of the denoising model 117 may be tuned over multiple iterations to increase (or maximize) the similarity between the noisy three-dimensional representation 182 of the sample molecule reconstructed by the denoising model 117 from the corrupted three-dimensional representation 184 of the sample molecule and the original noisy three-dimensional representation 182 of the sample molecule (e.g., to reduce (or minimize) the mean squared error (MSE)). Alternatively, the example in Figure 1B shows that the parameters of the denoising model 117 can be adjusted so that the denoising model 117 can reconstruct the embedding representation 186 generated by encoding the noisy three-dimensional representation 182 of the sample molecule from the corrupted embedding representation 188. For example, in the example shown in Figure 1B, the parameters of the denoising model 117 may be adjusted so that the denoising model 117 increases (or maximizes) the similarity between the embedding representation 186 reconstructed from the corrupted embedding representation 188 and the original uncorrupted embedding representation 186 of the three-dimensional representation 182 of the sample molecule (for example, the denoising model 117 reduces (or minimizes) an arbitrary loss function, such as the mean squared error (MSE), that quantifies the difference between the embedding representation 186 reconstructed from the corrupted embedding representation 188 and the original uncorrupted embedding representation 186 of the three-dimensional representation 182 of the sample molecule).

[0105] In some exemplary embodiments, training the molecular design computation model 115 may include determining a function 175 that can be parameterized by parameters (e.g., weights, biases, etc.) of a denoising model 117. For example, training the molecular design computation model 115, which may include tuning one or more parameters of the denoising model 117, may determine the function 175 (e.g., a score function) by tuning at least the corresponding parameters of the function 175. In some cases, the function 175 may approximate different densities across a data distribution of molecules exhibiting one or more desired properties, such that molecules exhibiting one or more properties are more likely to occupy a region of higher density in the data distribution. In the example shown in Figure 1A, this data distribution may be a noisy data distribution occupied by a noisy three-dimensional representation of molecules. Alternatively, in the example of the molecular design engine 110 shown in Figure 1B, this data distribution may be a noisy latent distribution occupied by a noisy embedding representation of the three-dimensional representation of molecules.

[0106] As described above, in some exemplary embodiments, overfitting of known molecules in the training dataset of the molecular design computation model 115, including the denoising model 117, can be avoided by training the denoising model 117 to approximate a noisy data distribution occupied by noisy three-dimensional representations of molecules, such as noisy voxelized representations, in the training dataset. An example of this is shown in Figure 1A, where the molecular design computation model 115 is trained to reconstruct a noisy three-dimensional representation 182 of a sample molecule, rather than a clean (or original) three-dimensional representation of the sample molecule. Once trained, as will be described in more detail below, the molecular design computation model 115 may generate a three-dimensional representation of an output molecule 158 by traversing the smoothed density of the noisy data distribution (i.e., iteratively sampling different regions of the data distribution) and sampling at least one updated three-dimensional representation 160. The noisy data distribution may be occupied by noisy three-dimensional representations of molecules exhibiting one or more desired properties. Therefore, the updated three-dimensional representation 160 may contain at least some noise that can be removed by applying a reconstruction model 118 to remove noise from the updated three-dimensional representation 160. The reconstruction model 118 may be a machine learning model trained to take a noisy three-dimensional representation 160 of a molecule as input and produce a corresponding denoised three-dimensional representation as output. In doing so, a three-dimensional representation of the output molecule 152 may be produced, which occupies the true data distribution of clean three-dimensional representations of molecules exhibiting one or more desired properties.

[0107] Alternatively, Figure 1B depicts another example of the molecular design engine 110 in which the molecular design calculation model 115 is trained to approximate a noisy latent distribution occupied by embedding representations of noisy three-dimensional representations of molecules exhibiting one or more desired properties. In this variation of the molecular design calculation model 115, the denoising engine 117 may be trained to reconstruct the embedding representation 186 of the noisy three-dimensional representation 182 of the sample molecule from a corrupted embedding representation 188. Once trained, the molecular design calculation model 115 may apply the denoising model 117 to denoise the embedding representation 154 generated by the encoder 111 encoding the three-dimensional representation of the input molecule 152. Denoising may include the denoising model 117 updating the embedding representation 154 over multiple consecutive sampling iterations to generate at least one updated embedding representation 156 during each sampling iteration. Doing so may be equivalent to the molecular design calculation model 115 selecting samples from a noisy latent distribution, and the molecular design calculation model 115 may continue to select samples from there until one or more criteria are met. In some cases, the updated embedding representation 156 may be decoded by, for example, the decoder 119 before the resulting noisy three-dimensional representation 158 is denoised by the reconstruction model 118 in order to generate a three-dimensional representation of the output molecule 162. As will be explained in more detail below, in this example of the molecular design engine 110, the molecular design calculation 115 may operate in a noisy latent space occupied by noisy embedding representations of the three-dimensional representations of molecules, rather than the noisy three-dimensional representations found in the discrete voxelized space. Furthermore, it should be understood that although the denoising model 117 and the reconstruction model 118 may in some cases share the same architecture (e.g., an artificial neural network (ANN)), the two models are trained to denoise different noises.For example, the denoising model 117 may be trained to denoise the three-dimensional representation of the input molecule 152 or its embedding representation 154 so that the resulting three-dimensional representation of the output molecule 162 matches the composition and / or conformation of a molecule exhibiting one or more desired properties (e.g., drug-like properties). In contrast, the reconstruction model 118 may be trained to remove noise added to smooth the density of known molecules available for training the molecular design calculation model 115.

[0108] Referring again to Figure 1B, the embedding representation 154 of the three-dimensional representation of the input molecule 152 can be generated by the encoder 111 encoding the three-dimensional representation of the input molecule 152. In some cases, the embedding representation 154 may be a lower-dimensional representation of the three-dimensional representation of the input molecule 152 generated by the encoder 111 reducing the dimensionality of the three-dimensional representation of the input molecule 152. For example, in some cases, the encoder 111 and decoder 119 may form an autoencoder including, for example, a variational autoencoder (VAE) such as a vector quantized variational autoencoder (VQ-VAE). The encoder 111 can generate the embedding representation 154 by reducing the dimensionality of at least the three-dimensional representation of the input molecule 152 (e.g., a voxelized representation). In this regard, reducing the dimensionality of the three-dimensional representation of the input molecule 152 may include, for example, downsampling, compressing, or reducing the dimensionality of the three-dimensional representation of the input molecule 152 by condensing at least some of the features (e.g., atomic density values) present in the three-dimensional representation of the input molecule 152, such that the resulting embedding representation 154 contains fewer features than the original three-dimensional representation of the input molecule 152, but those features still capture the same (or similar) information conveyed in the original three-dimensional representation of the input molecule 152. In the case of a voxelized representation of the input molecule 152, each feature present therein may correspond to an atomic density value associated with each voxel in the voxel grid representing the input molecule 152. For example, the voxelized representation of the input molecule 152 is If the voxel grid of TIFF2026518659000003.tif6170 is included, the voxelized representation of the input molecule 152 may include 32,000 features (or atomic density values). At least some of these 32,000 features may be condensed by the encoder 111 when generating the embedding representation 154. In doing so, the embedding representation 154 is condensed so that the denoising model 117 operates when generating the output molecule 162. It may include fewer features, such as those of TIFF2026518659000004.tif5170.

[0109] As mentioned, the embedding representation 154 of the three-dimensional representation of the input molecule 152 shown in Figure 1B may contain fewer features than the original three-dimensional representation of the input molecule 152. Furthermore, the embedding representation 154 may be generated by an encoder 111 that downsamples, compresses, or reduces the dimensionality of the three-dimensional representation of the input molecule 152. Doing so may be equivalent to the encoder 111 mapping the three-dimensional representation of the input molecule 152 from a high-dimensional discrete voxelized space to a lower-dimensional latent space. In some cases, the molecular design computation model 115 (e.g., denoising engine 117) may denoise the embedding representation 154, for example, over one or more iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling. During each iteration of gradient-based Markov Monte Carlo (MCMC) sampling, the molecular design computation model 115 (e.g., denoising model 117) may sample at least one updated embedding representation 156 from this lower-dimensional latent space. The fact that the embedding representation 154 may contain orders of magnitude fewer features than the original three-dimensional representation of the input molecule 152 means that the molecular design computation model 115 can apply the denoising model 117 to the embedding representation 154 to generate the output molecule 162, operating faster and with higher computational efficiency while achieving equivalent or better performance qualitatively and quantitatively. This can be particularly advantageous in applications such as computer drug design, which require the generation of a large number of candidate molecules within a short time period. In some cases, by reducing the dimensionality of the original three-dimensional representation of the input molecule 152, the denoising model 117 may be able to operate on and generate larger molecules (e.g., molecules containing more than 200 atoms) and larger quantities of molecules.

[0110] In some cases, the compactness of the embedding representation 154 relative to the three-dimensional representation of the input molecule 152 also means that fewer computational resources are required when working with the embedding representation 154. For example, if the denoising model 117 is a three-dimensional representation of the input molecule 152 To operate directly on TIFF2026518659000005.tif6170, the denoising model 117 may be implemented to include a large number of trainable parameters (e.g., 100 million parameters). Conversely, if the denoising model 117 is applied to operate instead on the embedding representation 154, the denoising model 117 can be implemented with far fewer trainable parameters. Implementing the denoising model 117 with fewer parameters may improve its performance, as a larger number of parameters can reduce the generalizability of the denoising model 117. For example, if the denoising model 117 includes a large number of features, but there are relatively few known molecules available to train the denoising model 117, the denoising model 117 may be particularly prone to overfitting. If the denoising model 117 is overfitted to known molecules in the training dataset, that is, if the denoising model 117 is overtrained on the training dataset (i.e., trained to learn a data distribution that is too concentrated on molecules in the training dataset), then the denoising model 117 may not be able to generalize. Generalization in this context refers to the ability to accurately denoise input molecule 152 when it is not one of the known molecules in the training dataset. Therefore, overfitting of the denoising model 117 may prevent the denoising model 117 from accurately denoising any molecule that is not one of the known molecules in the training data.

[0111] Referring again to Figure 1B, in the example of the molecular design engine 110 shown there, the updated embedding representation 156 generated by sampling from the latent voxelized space can be decoded by the decoder 119 before the resulting noisy three-dimensional representation 158 is denoised by the reconstruction engine 118. The decoding performed by the decoder 119 can map the updated embedding representation 156 from the latent space to the discrete space, while the subsequent denoising performed by the reconstruction model 118 can constitute a jump back from the noisy data distribution of the molecule to the true data distribution. For example, during each iteration of a gradient-based Markov chain Monte Carlo method (e.g., Langevin Markov chain Monte Carlo), the molecular design calculation model 115 may apply a denoising model 117 to sample from a noisy latent distribution of molecules exhibiting one or more desired properties. Sampling from the noisy latent distribution in this context may involve updating the embedding representation 154 of the three-dimensional representation (e.g., voxelized representation) of the input molecule 152. Doing so may be equivalent to selecting from the data distribution at least one updated embedding representation 156 that is denoised by the reconstruction model 118 and then decoded by the decoder 119 to generate a three-dimensional representation (e.g., a voxelized representation) of the output molecule 162. As will be described in more detail below, the molecular design calculation model 115 may further update the noisy embedding representation 156, for example, over multiple iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling, until one or more criteria are met.

[0112] If one or more criteria are met, the molecular design engine 110 may reconstruct the output molecule 162 based on at least a three-dimensional representation of the output molecule 162. For example, in some cases, the molecular design engine 110 may reconstruct the positions (e.g., coordinates) of atoms present in the output molecule 162 and one or more bonds between them, based on at least a three-dimensional representation of the output molecule 162. Reconstruction of the positions of atoms present in the output molecule 162 may be performed by identifying the maximum value (e.g., peak) of the predicted density included in the three-dimensional representation of the output molecule 162. In doing so, the molecular design engine 110 may determine another representation of the output molecule 162, including, for example, a one-dimensional representation of the output molecule 162 (e.g., a Simplified Molecular Input Line Notation (SMILES) string) or a two-dimensional representation of the output molecule 162 (e.g., a molecular graph). It should be understood that the output molecule 162 thus generated is more likely to exhibit one or more desired properties of the molecule in the data distribution. In particular, the denoising model 117 can generate output molecules 152 that exhibit a composition and / or conformation (or three-dimensional structure) that matches one or more desired properties. By working on an embedded representation 154 of the three-dimensional representation (e.g., voxelized representation) of the input molecule 152, the denoising model 117 can generate output molecules 162 more quickly and with less computational load.

[0113] As described above, in some exemplary embodiments, the molecular design calculation model 115 may be trained to reconstruct a noisy three-dimensional representation 182 of a sample molecule (e.g., a noisy voxelized representation of the sample molecule) that is generated by adding noise (e.g., Gaussian noise such as isotropic Gaussian noise) to the three-dimensional representation of the sample molecule. In the example shown in Figure 1A, the molecular design calculation model 115 may be trained to denoise the corrupted three-dimensional representation 182 of the sample molecule by correcting the corrupted three-dimensional representation 184 of the sample molecule to at least reconstruct the noisy three-dimensional representation 184 of the sample molecule. Alternatively, Figure 1B shows an example of a molecular design engine 110 in which the molecular design calculation model 115 is trained to reconstruct an embedded representation 186 of the noisy three-dimensional representation 182 of the sample molecule. As shown in Figure 1B, in some cases, the molecular design calculation model 115 may denoise the corrupted embedding representation 186 by correcting the corrupted embedding representation 188, which may be generated by the corrupted engine 121 adding noise (e.g., Gaussian noise) to the embedding representation 184 of the three-dimensional representation 182 of the sample molecule. As described above, the embedding representation 184 may be generated by the encoder 111 downsampling the noisy three-dimensional representation 182 of the sample molecule (e.g., a noisy voxelized representation) or reducing its dimensionality. However, as described above, downsampling the noisy three-dimensional representation 182 of the sample molecule (e.g., a noisy voxelized representation) may be optional, which may be the case if the encoder 111 implements an identity function. Thus, in some cases, the embedding representation 184 may contain the same amount of features as the original noisy three-dimensional representation 182 of the sample molecule (e.g., a noisy voxelized representation). In those cases, the embedded representation 154 can capture the same information present in the original three-dimensional representation (e.g., voxelized representation) of the input molecule 152 without the features present therein being condensed.In other words, the encoding of the three-dimensional representation (e.g., voxelized representation) of the input molecule 152 may be any operation, even if the example molecular design system 100 shown in Figure 1B includes an encoder 111.

[0114] Figure 2 depicts a flowchart illustrating an example of a process 200 for machine learning correspondence generation of three-dimensional molecules in voxelized space, according to several exemplary embodiments. Referring to Figures 1A–1B and 2, process 200 may be carried out by a molecular design engine 110 to train and apply a molecular design computation model 115 to generate an output molecule 162 by denoising a three-dimensional representation, such as a voxelized representation, of an input molecule 152. For example, in some cases, the molecular design computation model 115 may be trained on a noisy three-dimensional representation 182 of a sample molecule, generated by adding noise to the original three-dimensional representation of the sample molecule, so that the molecular design computation model 115 is trained to approximate a noisy data distribution with smoother density transitions. Figure 1A shows one variation in which the molecular design computation model 115 is trained on a corrupted three-dimensional representation 184 of a sample molecule, which can be generated by adding additional noise to the noisy three-dimensional representation 182 of the sample molecule without any downsampling or compression. Alternatively, Figure 1B shows another variation in which the molecular design computation model 115 is trained on a corrupted embedding representation 186 of a noisy three-dimensional representation 182 of the sample molecule. This corrupted embedding representation 188 may be generated by adding noise to the embedding representation 154 of the noisy three-dimensional representation 182 of the sample molecule, which is produced by the encoder 111 downsampling (or compressing) features present in the noisy three-dimensional representation 182 of the sample molecule (e.g., a noisy voxelized representation). In other words, it should be understood that the molecular design computation model 115 may be trained to operate in either a noisy discrete voxelized space occupied by the noisy three-dimensional representation of the molecule, or alternatively, a noisy latent voxelized space occupied by the embedding representation of the noisy three-dimensional representation of the molecule.

[0115] In 202, the molecular design engine 120 may generate a training dataset containing multiple corrupted sample molecules. In some exemplary embodiments, generating a training dataset may include the training engine 120 generating a training dataset containing multiple corrupted sample molecules. This training dataset may then be used to train the molecular design computation model 115 to approximate the data distribution of molecules exhibiting one or more desired properties (e.g., drug-like properties). In some cases, each corrupted sample molecule may be a noisy three-dimensional representation of a known molecule that is further corrupted with additional noise (e.g., Gaussian noise such as isotropic Gaussian noise). For example, Figure 1A shows an example of this in which the corruption engine 121 generates a corrupted three-dimensional representation 184 of a sample molecule by adding additional noise to a noisy three-dimensional representation 182 of the sample molecule. As will be described in more detail below, the molecular design calculation model 115 (e.g., denoising model 117) may be trained to approximate a noisy data distribution with smoother density transitions by being trained to reconstruct a noisy three-dimensional representation 182 of a sample molecule from a corrupted three-dimensional representation 184 of the sample molecule. Alternatively, Figure 1B shows another example in which the training engine 120 generates a corrupted sample molecule in the training dataset so that it contains a corrupted embedding representation of the noisy three-dimensional representation 182 of the sample molecule. For example, in some cases, the corruption engine 121 may generate a corrupted embedding representation 188 by adding noise (e.g., Gaussian noise such as isotropic Gaussian noise) to the embedding representation 186 of the noisy three-dimensional representation 182 of the sample molecule (e.g., a noisy voxelized representation). In some cases, the training engine 120 may further augment the training dataset by applying one or more transformations to the noisy three-dimensional representation 182 of the sample molecule (e.g., a voxelized representation), including, for example, translation (e.g., by shifting the center of the sample molecule in each of the three dimensions by sampling a uniform shift), rotation (e.g., by uniformly sampling three Euler angles), and reflection.

[0116] By training the molecular design computation model 115 based on a noisy three-dimensional representation 182 of sample molecules, the occurrence of overfitting and mode collapse can be mitigated, typically when the molecular design computation model 115 is based on a disproportionately small number of known molecules (e.g., PubChem dataset, QM9 molecular dataset, Molecular Geometry Ensemble (GEOM) drug dataset, etc.) with high-dimensional data distribution This occurs when the model is trained to approximate TIFF2026518659000006.tif5170.

[0117] In the example of molecular design calculation model 115 shown in Figures 1A and 1B, the molecular design calculation model 115 may include a denoising model 117. In this case, training the molecular design calculation model 115 may include training the denoising model 117 to approximate a noisy data distribution, or possibly a noisy latent distribution, which either of which exhibits a smoother density transition and is more efficient to sample than the true data distribution. In some cases, the denoising model 117 may be an artificial neural network (ANN), in which case training the denoising model 117 may include tuning one or more parameters of the artificial neural network (ANN) (e.g., weights, biases, etc.). By doing so, the parameters of function 175 can also be determined such that function 175 outputs a value indicating the likelihood that molecules exhibiting one or more desired properties are located at a particular position in the data distribution. For example, in some cases, function 175 may be a score function whose output is a value (e.g., a score) that indicates transitions between different density regions of the data distribution, including transitions between, for example, higher density regions that are more likely to be occupied by molecules exhibiting one or more desired properties and lower density regions of the data distribution that are less likely to be occupied by molecules exhibiting one or more desired properties. In some cases, the denoising model 117 may be trained to reconstruct a noisy three-dimensional representation 182 of sample molecules, for example, to avoid overfitting the denoising model 117 to a relatively small number of known molecules in the training dataset available to characterize the data distribution.

[0118] To illustrate further, TIFF2026518659000007.tif6170 shows the true data distribution of voxelized representations of molecules exhibiting one or more desired properties. TIFF2026518659000008.tif6170 can exhibit a smoother energy landscape. This shows a corresponding noisy data distribution that can be sampled more efficiently than TIFF2026518659000009.tif6170. In some cases, TIFF2026518659000010.tif6170 may be unknown, and this is because the denoising model 117 is based on a training dataset of known molecules from the true data distribution. This means that it can be trained to approximate TIFF2026518659000011.tif6170. To avoid overfitting the denoising model 117 to the training dataset, the denoising model 117 is trained to: It may be trained to approximate TIFF2026518659000012.tif6170. In some cases, TIFF2026518659000013.tif6170 may also be obtained by convolving with a Gaussian kernel (for example, an isotropic Gaussian kernel). Doing so means From TIFF2026518659000014.tif6170 By adding TIFF2026518659000015.tif5170 This may be equivalent to generating TIFF2026518659000016.tif5170. Given the above formula, It can be sampled from TIFF2026518659000017.tif6170.

number

[0119] In this way TIFF2026518659000019.tif6170 A clean (or original) version with no additions. TIFF2026518659000020.tif5170 still retains some of the existing structural information, TIFF2026518659000021.tif6170 can be smoothed. Clean TIFF2026518659000022.tif5170 If it is Gaussian (e.g., isotropic Gaussian), then clean TIFF2026518659000023.tif5170 is shown as equation (1) below. By applying TIFF2026518659000024.tif5170, the corresponding noisy TIFF2026518659000025.tif5170 can be restored directly. TIFF2026518659000026.tif5170 Noisy By removing TIFF2026518659000027.tif5170, it acts as a noise remover, resulting in a clean file. Please understand that TIFF2026518659000028.tif5170 can be restored.

number

[0120] As will be explained in more detail below, once the denoising model 117 is trained, the molecular design calculation model 115 applies the denoising model 117 to perform the "walk jump" generation process. It is possible to generate output molecules exhibiting one or more desired properties of the molecules in TIFF2026518659000037.tif5170. For example, in some cases, the denoising model 117 may over multiple consecutive sampling iterations. It can be sampled from TIFF2026518659000038.tif5170, and each of them has at least one noise reduction model 117. By denoising TIFF2026518659000039.tif5170 This includes selecting at least one sample from TIFF2026518659000040.tif5170. In some cases, The sampling of TIFF2026518659000041.tif5170 is such that the sample selected during one sample iteration is TIFF2026518659000042.tif5170 So that the sample selected during another sample iteration originates from a different position, It may be induced by TIFF2026518659000043.tif5170. This traverse of TIFF2026518659000044.tif5170 is the so-called "walking" part of the generation process.

[0121] In some cases, Instead of freely sampling from TIFF2026518659000045.tif6170, Based on TIFF2026518659000046.tif5170 TIFF2026518659000047.tif5170 may be restricted. Therefore, in some cases, each consecutive sample is, As indicated by the score output by TIFF2026518659000048.tif5170, A gradually increasing density region may be selected from TIFF2026518659000049.tif5170. Here again, as mentioned above, Traversing TIFF2026518659000050.tif5170 means TIFF2026518659000051.tif5170 could be interpreted as "walking". Corresponding noise Clean from TIFF2026518659000052.tif5170 This could constitute a "jump" back to TIFF2026518659000053.tif5170. In some cases, The "Jump" button to return to TIFF2026518659000054.tif5170 is Clean corresponding to TIFF2026518659000055.tif5170 Restoring TIFF2026518659000056.tif5170 may be achieved by applying a noise reduction engine such as denoising engine 117. For example, in some cases, a clean TIFF2026518659000057.tif5170 Noise reduction engine 117 is noisy It may be restored by applying TIFF2026518659000058.tif5170.

[0122] In some exemplary embodiments, the training engine 120 may generate each corrupted sample molecule in the training dataset by adding noise to the embedding representation of the noisy three-dimensional representation of the sample molecule. For example, Figure 1B shows that, in some cases, the corruption engine 121 may generate a corrupted embedding representation 188 by adding noise (e.g., Gaussian noise such as isotropic Gaussian noise) (not directly to the noisy three-dimensional representation 182 of the sample molecule) to the embedding representation 186 of the noisy three-dimensional representation 182 of the sample molecule (e.g., a noisy voxelized representation). Figure 1B further shows that the embedding representation 186 may be generated by the encoder 111 downsampling (or compressing) the noisy three-dimensional representation 182 of the sample molecule. Downsampling (or compressing) the noisy three-dimensional representation 182 of the sample molecule may reduce the dimensionality of the noisy three-dimensional representation 182 of the sample molecule. For example, the noisy three-dimensional representation 182 of the sample molecule contains 32,000 features (or atomic density values) If TIFF2026518659000059.tif5170 is a voxel grid, downsampling (or compression) will result in 64 features (or atomic density values) being included. TIFF2026518659000060.tif5170 may result in a voxel grid. Therefore, downsampling (or compression) of the noisy three-dimensional representation of the sample molecule 182 can increase the overall speed and efficiency of the generation process. In some cases, a large number of candidate molecules, such as tens of thousands or even millions of candidate molecules, may be generated in a short period of time to support low-yield applications such as computer drug design where most candidate molecules cannot be successfully synthesized in the laboratory. Downsampling (or compression) of the three-dimensional representation of the sample molecule 182 can also enable the generation of larger-sized molecules (e.g., molecules containing more than 200 atoms) that would be very cumbersome to operate if they were retained in their original three-dimensional representation without any downsampling (or compression).

[0123] In step 204, the molecular design engine 110 may train the molecular design computation model 115 by applying the molecular design computation model 115 to reconstruct the three-dimensional representation of each corrupted sample molecule in the training dataset from the corrupted three-dimensional representation of the sample molecule. In some exemplary embodiments, step 204 of training the molecular design computation model 115 may include training a denoising model 117 to approximate a noisy data distribution (or noisy latent distribution) of molecules exhibiting one or more desired properties, based on at least the training dataset. For example, in the example shown in Figure 1A, the denoising model 117 may be trained to reconstruct a noisy three-dimensional representation 182 (e.g., a voxelized representation) of the sample molecule from a corrupted three-dimensional representation 184 of the sample molecule. In the example shown in Figure 1B, the denoising model 117 may be trained to reconstruct an embedding representation 186 of the noisy three-dimensional representation 182 of the sample molecule from a corrupted embedding representation 188. In each example, it should be understood that the denoising engine 117 may be trained on a noisy three-dimensional representation 182 of the sample molecule rather than a clean three-dimensional representation of the sample molecule, in order to approximate a noisy data distribution that exhibits smoother density transitions than the true data distribution.

[0124] In the example shown in Figure 1A, training the molecular design calculation model 115 may include adjusting one or more parameters of the denoising model 117 (e.g., weights, biases, etc.) to reduce (or minimize) the difference (e.g., mean squared error (MSE)) between the noisy three-dimensional representation 182 of the molecule reconstructed by the denoising model 117 and the original noisy three-dimensional representation 182 of the sample molecule. Furthermore, training the denoising model 117 may include determining a function 175 that is parameterized by the parameters of the denoising model 117 (e.g., weights, biases, etc.). For example, in some cases, the function 175 may be a scoring function that outputs a value (e.g., a score) that indicates a local change (or gradient) in the density of the data distribution. Therefore, in some cases, function 175 may output a first value (e.g., a first score) indicating a first local change in the density of the data distribution at a first position occupied by the first numerator, and a second value (e.g., a second score) indicating a second local change in the density of the data distribution at a second position occupied by the second numerator. For example, the first value (e.g., the first score) may indicate a more positive local change (e.g., an increase or a smaller decrease) in the density of the data distribution at the first position of the first numerator, while the second value (e.g., the second score) may indicate a less positive local change (e.g., a smaller increase or decrease) in the density of the data distribution at the second position of the second sample. If function 175 is a score function, sampling of the data distribution may be induced by the value (e.g., score) output by function 175. As described in more detail below, sampling of the data distribution can be guided by function 175 such that the sample (or molecule) is selected from a region of the data distribution with an increasingly higher density, where it is more likely that the sample (or molecule) is occupied by one or more molecules exhibiting the desired properties.

[0125] In the example shown in Figure 1B, training the denoising model 117 may include adjusting one or more parameters of the denoising model 117 (e.g., weights, biases, etc.) to reduce (or minimize) the difference (e.g., mean squared error (MSE)) between the original undamaged embedding representation 186 and the noisy three-dimensional representation 182 of the sample molecules reconstructed by the denoising model 117 from the damaged embedding representation 188. The parameters of function 175 can also be adjusted in this way. Again, if function 175 is a score function, the parameters of function 175 may be adjusted so that function 175 outputs a higher value (e.g., a higher score) for a first molecule occupying a first position in the data distribution that exhibits a more positive local change in the density of the data distribution (e.g., a positive gradient indicating a transition from a lower density region to a higher density region of the data distribution) than for a second molecule occupying a second position in the data distribution that exhibits a less positive local change in the density of the data distribution.

[0126] In some exemplary embodiments, the molecular design engine 110 may train a molecular design computation model 115, including a denoising model 117, by approximating the function 175 by performing gradient-based Markov chain Monte Carlo (MCMC) sampling, such as Markov chain Monte Carlo (MCMC) sampling by Langevin Dynamics. In some cases, the function 175 may output a value (e.g., a score) that indicates transitions between different density regions of the data distribution. For example, as a score function, the value (e.g., score) output by the function 175 for each molecule may indicate a local change (or gradient) in density at the corresponding location in the data distribution. In the example shown in Figure 1A, gradient-based Markov chain Monte Carlo (MCMC) sampling to determine function 175 may include adjusting the parameters of the denoising model 117 (e.g., weights, biases, etc.) and the parameters of function 175 over multiple iterations to increase (or maximize) the similarity between the noisy three-dimensional representation 182 of the sample molecule reconstructed by the denoising model 117 and the original three-dimensional representation 182 of the sample molecule (e.g., by reducing (or minimizing) the mean squared error (MSE)). In the example shown in Figure 1B, one or more parameters of the denoising model 117 and the parameters of function 175 may be adjusted over multiple iterations of gradient-based Markov chain Monte Carlo (MCMC) to increase (or maximize) the similarity between the embedding representation 186 of the noisy three-dimensional representation 182 of the sample molecule reconstructed by the denoising engine 117 from the corrupted embedding representation 188 and the original uncorrupted embedding representation 186 (e.g., by reducing (or minimizing) the mean squared error (MSE)).

[0127] As described above, the denoising model 117 may be trained to reconstruct a noisy three-dimensional representation 182 (e.g., a noisy voxelized representation) of the sample molecule in order to avoid overfitting the denoising model 117 to known molecules available for training the denoising model 117. When a relatively small number of known molecules characterizing the high-dimensional data distribution are available, directly training the denoising model 117 based on the known molecules may result in an overly sawtooth energy landscape with abrupt gradient changes between regions occupied by the known molecules. Sampling from the data distribution while steep gradients are induced may hinder a proper exploration of the data distribution, as the steepness of the gradients may confine sampling to regions close to the known molecules. In contrast, training the denoising model 117 based on a noisy three-dimensional representation 182 (e.g., a noisy voxelized representation) of the sample molecule results in smoother density transitions, with gentler gradients in function 175, allowing for a more efficient exploration of the data distribution when sampling from there.

[0128] In 206, the molecular design engine 110 may apply a trained molecular design computation model 115 to generate an output molecule by denoising at least the voxelized representation of the input molecule. In some exemplary embodiments, the molecular design computation model 115 may generate an output molecule 162 by updating the three-dimensional representation (e.g., voxelized representation) of the input molecule 152, or optionally its embedding representation 154, using a denoising model 117, guided at least by a function 175. For example, the molecular design computation model 115 in Figure 1A may directly update the three-dimensional representation (e.g., voxelized representation) of the input molecule 152 without any downsampling or compression. Alternatively, the molecular design computation model 115 in Figure 1B may update the embedding representation 154 of the three-dimensional representation of the input molecule 152, which may be generated by the encoder 111 downsampling (or compressing) the three-dimensional representation (e.g., voxelized representation) of the input molecule 152. By doing so, the dimensionality (or number of features) of the three-dimensional representation (e.g., voxelized representation) of the input molecule 152 may be reduced, and as a result, the resulting embedding representation 154 may be more compact than the original (or uncompressed) three-dimensional representation of the input molecule 152. For example, the original three-dimensional representation (e.g., voxelized representation) of the input molecule 152 contains 32,000 features (or atomic density values). TIFF2026518659000061.tif5170 may contain a voxel grid, but embedding representation 154 contains 64 features (or atomic density values). TIFF2026518659000062.tif5170 may include a voxel grid. It should be understood that the encoder 111 may be trained to downsample (or compress) the voxelized representation of the input molecule 152 such that the resulting embedding representation 154 conveys the same (or similar) information as the voxelized representation of the input molecule 152 in its original (or uncompressed) form. In some cases, the encoder 111 may be part of an autoencoder (e.g., a variational autoencoder (VAE), such as a vector quantized variational autoencoder (VQ-VAE)), which also includes a decoder 119. In some cases, the encoder 111 may be trained to generate the embedding representation 154 such that the decoder 119 can reconstruct the original voxelized representation of the input molecule 152 by decoding at least the embedding representation 154.

[0129] In some cases, the molecular design calculation model 115 may denoise the input molecule 152 by updating at least the three-dimensional representation of the input molecule 152, or alternatively, the embedding representation 154 of the three-dimensional representation of the input molecule 152. As described above, in some cases, the three-dimensional representation of the input molecule 152 may be a voxelized representation of the input molecule 152, where the types and positions of atoms present in the input molecule 152 are represented as atom-centered continuous (e.g., Gaussian) atomic densities. For example, in some cases, the voxelized representation of the input molecule 152 may be associated with values ​​that indicate the atomic density at each corresponding position. Includes voxels of TIFF2026518659000063.tif5170 TIFF2026518659000064.tif5170 may include a voxel grid. In some cases, the atomic density associated with a single voxel is The values ​​may be within a range such as TIFF2026518659000065.tif5170, where the atomic density at the lower end of the range indicates that the voxel is further away from any atom in the input molecule 152, and the atomic density at the upper end of the range indicates that the voxel is closer to the center of the atom in the input molecule 152. Furthermore, the voxelized representation of the input molecule 152 may include multiple channels, each corresponding to a different type of atom that may be present in the input molecule 152. Thus, the voxelized representation of the input molecule 152 may represent the type and position of atoms present in the input molecule 152 as a continuous (e.g., Gaussian) atomic density across one or more channels.

[0130] In some exemplary embodiments, denoising of the input molecule 152 may include updating the three-dimensional representation of the input molecule 152, or alternatively, the embedding representation 154 of the input molecule 152. If the molecular design calculation model 115 operates on the three-dimensional representation of the input molecule 152, or if the embedding representation 154 is generated without any downsampling (or compression) of the three-dimensional representation of the input molecule 152, denoising may include updating the atomic density of one or more voxels in at least one channel of the noisy voxelized representation of the input molecule 152. Doing so may be equivalent to adding, removing, and / or rearranging one or more atoms of different atomic types in the input molecule 152. For example, increasing (or decreasing) the atomic density of one or more voxels in one channel of the noisy voxelized representation of the input molecule 152 may be equivalent to adding (or removing) atoms of the corresponding type to the input molecule 152. Alternatively and / or additionally, decreasing the atomic density of the first voxel while increasing the atomic density of the second voxel may be equivalent to rearranging atoms from a first position in the first voxel to a second position in the second voxel.

[0131] Alternatively, if the molecular design calculation model 115 operates on an embedded representation 154 of the three-dimensional representation (e.g., voxelized representation) of the input molecule 152, denoising may include updating the values ​​of the voxels present in the embedded representation 154. As described above, the embedded representation 154 may be generated by the encoder 111 condensing at least some of the features (e.g., atomic density values) present in the voxelized representation of the input molecule 152. The embedded representation 154 may contain fewer features than the original voxelized representation of the input molecule 152, but may still convey the same (or similar) information as the original three-dimensional representation of the input molecule 152. Therefore, denoising of the embedded representation 154 may include updating one or more values ​​present in the embedded representation 154, at least some of which may represent multiple features (or atomic density values) from the original voxelized representation of the input molecule 152.

[0132] Updating the three-dimensional representation or its embedding representation 154 of the input molecule 152 as described above may involve selecting a sample (or updated molecule) from a noisy data distribution (or noisy latent distribution) of molecules exhibiting one or more desired properties. In the case of gradient-based Markov chain Monte Carlo (MCMC) sampling, the update may be induced by the output of function 175 (e.g., the score output by function 175) such that the sample (or updated molecule) selected during each successive sampling iteration originates from a region of the noisy data distribution with an increasingly higher density, where it is more likely to be occupied by molecules exhibiting one or more desired properties.

[0133] To illustrate further, in some cases, the three-dimensional representation of the input molecule 152 or its embedding representation 154 may undergo a first update and a second update. Doing so may be equivalent to selecting a first sample (or first updated molecule) and a second sample (or second updated molecule) from a noisy data distribution. In some cases, once a first sample (or first updated molecule) and a second sample (or second updated molecule) are selected from a noisy data distribution (or noisy latent distribution), the molecular design calculation model 115 may apply function 175 to determine a value (e.g., a score) that indicates the likelihood that each sample (or updated molecule) is within the noisy data distribution (or noisy latent distribution). If function 175 is a score function, for example, a higher value (e.g., a lower score) may indicate that the sample (or updated molecule) is selected from a region of the noisy data distribution exhibiting a more positive local change in density (e.g., an increase or a smaller decrease), or similarly, that the sample (or updated molecule) has a higher likelihood of being within the noisy data distribution. Thus, in some cases, after selecting the first sample (or first updated molecule) and the second sample (or second updated molecule), the molecular design calculation model 115 may apply the denoising model 117 to continue updating the three-dimensional representation or its embedding representation 154 of the input molecule 152 to select additional samples (or further updated molecules) from increasingly dense regions of the noisy data distribution until, for example, a sample (or updated molecule) exhibiting a threshold likelihood of being within the noisy data distribution (or latent distribution) is selected. For example, in some cases, the denoising model 117 may be applied to further modify the three-dimensional representation or embedding representation 154 of the input molecule 152 (or the first updated molecule) having the first update, if the three-dimensional representation or embedding representation 154 of the input molecule 152 (or the first updated molecule) having the first update is selected from a denser region of the data distribution.Doing so may be analogous to traversing a noisy data distribution (or noisy latent distribution) and sampling from regions of progressively higher density in the data distribution. If the denoising model 117 modifies the embedding representation 154 of the three-dimensional representation of the input molecule 152, the denoising model 117 may be operating in a noisy latent space that may reflect similarities (or differences) in the types and positions of atoms in molecules where the distance between two or more embedding representations is different. Abrupt transitions in density where the true data distribution of molecules exhibiting one or more desired properties exists can be smoothed by adding noise. Since the denoising model 117 is trained to approximate the data distribution of molecules exhibiting a particular desired property (e.g., drug-like properties), the updates made to the embedding representation 154 when denoising the input molecule 152 may coincide with the types and positions of atoms found in molecules exhibiting one or more desired properties. Therefore, the same desired properties may also exist in the output molecule 162 generated by applying the denoising model 117 to the molecular design calculation model 115 to denoise the input molecule 152.

[0134] Figure 3A depicts a flowchart illustrating an example of a process 300 for training a molecular design computation model 115 according to several exemplary embodiments. Referring to Figures 1-2 and 3A, process 300 may perform operation 204 of process 200 shown in Figure 2. In some cases, process 300 may be performed by a molecular design engine 110 that trains the molecular design computation model 115, for example, a denoising model 117, to approximate the noisy data distribution of a noisy three-dimensional representation (e.g., a noisy voxelized representation) of a molecule exhibiting one or more desired properties. In some cases, as will be described in more detail below, the molecular design computation model 115, including the denoising model 117, may be trained to approximate a noisy data distribution instead of a true data distribution in order to avoid overfitting the molecular design computation model 115 to known molecules available for training the molecular design computation model 115. As described in more detail below, in some cases, the molecular design calculation model 115, including the denoising model 117, may be trained through gradient-based Markov chain Monte Carlo (MCMC) sampling, such as Markov chain Monte Carlo (MCMC) sampling with Langevin dynamics.

[0135] In step 302, the molecular design engine 110 may apply a molecular design computation model having a first adjustment for denoising corrupted sample molecules and generating a first updated molecule. In some exemplary embodiments, step 302 may include the molecular design engine 110 training a molecular design computation model 115, including, for example, a denoising model 117, to approximate a data distribution of three-dimensional representations (e.g., voxelized representations) of one or more molecules exhibiting the same desired properties, from which candidate molecules exhibiting the same desired properties can be generated by sampling. In some cases, the molecular design computation model 115 may be trained to approximate the aforementioned data distribution based on a training dataset of corrupted sample molecules, each generated based on a noisy three-dimensional representation (e.g., voxelized representation) of sample molecules (e.g., known molecules) from the data distribution. An example of this is shown in Figure 1A, where the molecular design calculation model 115 (e.g., denoising model 117) is trained to reconstruct a noisy three-dimensional representation 182 of the sample molecule from a corrupted three-dimensional representation 184 of the sample molecule generated by the corruption engine 121. In some cases, instead of being trained to directly reconstruct the noisy three-dimensional representations of the sample molecule (e.g., voxelized representations), the molecular design calculation model 115 may be trained based on corrupted embedding representations of those three-dimensional representations (e.g., voxelized representations). This is shown in Figure 1B, where the molecular design calculation model 115 (e.g., denoising model 117) is trained to reconstruct an embedding representation 186 of the noisy three-dimensional representation 182 of the sample molecule from a corrupted embedding representation 188 generated by the corruption engine 121.

[0136] In some exemplary embodiments, training the molecular design computation model 115 may include applying a denoising model 117 to denoise a corrupted three-dimensional representation (e.g., a voxelized representation) or a corrupted embedding representation of each sample molecule. Figure 1A shows an example in which training the molecular design computation model 115 includes adjusting the parameters of the denoising model 117 (e.g., weights, biases, etc.) such that the difference (e.g., mean squared error (MSE)) between the noisy three-dimensional representation (e.g., a voxelized representation) of the sample molecule and the one reconstructed by the denoising model 117 denoising the corrupted three-dimensional representation of the sample molecule is progressively reduced, for example, over multiple iterations. Alternatively, in the example shown in Figure 1B, the molecular design calculation model 115 may be trained by adjusting the parameters of the denoising model 117 (e.g., weights, biases, etc.) so that the difference (e.g., mean squared error) between the embedding representation of a noisy three-dimensional representation of the sample molecule and the embedding representation that the denoising model 117 reconstructs from the corresponding corrupted embedding representation is progressively reduced over multiple iterations. In some cases, the parameters of the denoising model 117 (e.g., weights, biases, etc.) may undergo different adjustments before further adjustments are made for adjustments that result in a lower difference (e.g., mean squared error (MSE)). For example, in some cases, the first adjustment may be made to the parameters of the denoising model 117 (e.g., weights, biases, etc.) before the denoising model 117 with the first adjustment is applied to denoise a corrupted three-dimensional representation of the sample molecule or a corrupted embedding representation of the three-dimensional representation of the sample molecule and generate at least a first updated molecule. In some cases, the first updated molecule may be an updated three-dimensional representation (e.g., a voxelized representation) of the first molecule, or alternatively, an updated embedded representation of the three-dimensional representation (e.g., a voxelized representation) of the first molecule.

[0137] In some cases, a corrupted three-dimensional representation of a sample molecule or a corrupted embedding representation of a three-dimensional representation may be denoised by updating one or more atomic density values ​​that represent the types and locations of atoms present in the sample molecule. If the corrupted embedding representation is generated by downsampling (or compressing) the three-dimensional representation of the sample molecule, at least some of the updated values ​​may condense several features (or atomic density values) from the original three-dimensional representation of the sample molecule (e.g., a voxelized representation). As described in more detail below, a denoising model 117 having a second adjustment (instead of the first adjustment) may be applied to denoise a corrupted three-dimensional representation of a sample molecule (e.g., a corrupted voxelized representation) or a corrupted embedding representation of a three-dimensional representation of a sample molecule and generate at least a second updated molecule. Further adjustments may be made to the denoising model 117 having either the first or second adjustment. By doing so, the denoising model 117 can be trained to approximate a noisy data distribution, or in some cases a noisy latent distribution, which exhibits smoother density transitions to support more efficient sampling because there are no steep gradient changes that confine sampling to a region immediately adjacent to the sample molecules that form the basis of the training dataset.

[0138] In some exemplary embodiments, training the denoising model 117 may further include determining a function 175. As described above, in some cases, the function 175 may be a score function parameterized by the parameters of the denoising model 117 (e.g., weights, biases, etc.). Thus, in some cases, training the molecular design computation model 115, which involves tuning the parameters of 117, may also include tuning the parameters of the function 175. For example, in some cases, the function 175 may be determined by performing gradient-based Markov chain Monte Carlo (MCMC) sampling (e.g., Langevin Markov chain Monte Carlo (MCMC) sampling) to approximate the gradient of a noisy data distribution (or noisy latent distribution). Doing so may involve tuning the parameters of the function 175 over one or more iterations so that the function 175 outputs a value (e.g., a score) that indicates a change in local density in the noisy data distribution (or noisy latent distribution). If function 175 is a scoring function, the parameters of function 175 may be adjusted so that function 175 assigns higher values ​​(e.g., higher scores) to samples such as a three-dimensional representation of a molecule or its embedding representation from locations exhibiting more positive local changes in density (e.g., increases or decreases) than to samples from locations exhibiting less positive local changes in density (e.g., decreases or smaller increases). Thus, once the denoising model 117 is trained, function 175 may output values ​​(e.g., scores) that distinguish between samples from higher-density regions of the noisy data distribution (or noisy latent distribution) (e.g., three-dimensional representations, embedding representations of three-dimensional representations, etc.) and samples sampled from lower-density regions of the noisy data distribution.

[0139] In step 304, the molecular design engine 110 may apply a molecular design calculation model having a second adjustment to denoise the corrupted sample molecule and generate a second updated molecule. In some exemplary embodiments of step 304, once a denoising model 117 having a first adjustment is applied to generate at least a first updated molecule, a denoising model 117 having a second adjustment may be applied to generate at least a second updated molecule, for example, an updated three-dimensional representation (e.g., an updated voxelized representation of the second molecule or an updated embedded representation of the three-dimensional representation of the second molecule). It should be understood that the first and second adjustments may involve different changes to the parameters of the denoising model 117 (e.g., weights, biases, etc.). Thus, to denoise the corrupted three-dimensional representation 184 of the sample molecule or the corrupted embedded representation 186 of the noisy three-dimensional representation 182 of the sample molecule, Applying a denoising model 117 with two adjustments may result in an updated molecule that differs from applying a denoising model 117 with a first adjustment to denoise a corrupted three-dimensional representation 182 of the sample molecule or a corrupted embedding representation 186 of a noisy three-dimensional representation 182 of the same sample molecule. As will be described in more detail below, training the denoising model 117 may include further adjustments of the denoising model 117 with either the first or second adjustment, depending on the difference (e.g., mean squared error (MSE)) present in the noisy three-dimensional representation 182 of the sample molecule (Figure 1A) or the embedding representation 184 of the noisy three-dimensional representation 182 of the sample molecule (Figure 1B) reconstructed by the denoising model 117.

[0140] In step 306, the molecular design engine 110 may determine that the first updated molecule is more similar to the sample molecule than the second updated molecule. In some exemplary embodiments, step 306 may also include the molecular design engine 110 selecting the denoising model 117 with the first adjustment for further adjustment in subsequent iterations if the first updated molecule generated by the denoising model 117 with the first adjustment is more similar to the noisy three-dimensional representation (or its embedding representation) of the sample molecule than the second updated molecule generated by the denoising model 117 with the second adjustment (e.g., exhibits a lower mean squared error (MSE)). For example, in Figure 1A, the first updated molecule may be an updated three-dimensional representation of the first molecule that has a smaller difference (e.g., a lower mean squared error (MSE)) to the noisy three-dimensional representation 182 of the sample molecule than the second updated molecule. In Figure 1B, the first updated molecule may be an updated embedding representation of the three-dimensional representation of the first molecule, having a smaller difference (e.g., a lower mean squared error (MSE)) from the embedding representation 186 of the noisy three-dimensional representation 182 of the sample molecule than the second updated molecule.

[0141] The fact that the first updated molecule is more similar to the noisy three-dimensional representation 182 (or its embedded representation 186) of the sample molecule than the second updated molecule may indicate that the denoising model 117 with the first adjustment is better at reconstructing the noisy three-dimensional representation 182 (or its embedded representation 186) of the sample molecule than the denoising model 117 with the second adjustment. Therefore, the denoising model 117 with the first adjustment may better approximate the noisy data distribution (or noisy latent distribution) of molecules exhibiting one or more desired properties than the denoising model 117 with the second adjustment. Therefore, in some cases, the molecular design engine 110 may select the denoising model 117 with the first adjustment instead of the denoising model 117 with the second adjustment to undergo one or more additional iterations of adjustment.

[0142] In 308, the molecular design engine 110 may further refine the molecular design calculation model having the first adjustment instead of the second adjustment until one or more criteria are met. In some exemplary embodiments, if the first updated molecule generated by the denoising model 117 having the first adjustment is more similar to the noisy three-dimensional representation 182 (or its embedding representation 186) of the sample molecule than the second updated molecule generated by the denoising model 117 having the second adjustment (e.g., has a lower mean squared error (MSE)), the molecular design engine 110 may further refine the denoising model 117 having the first adjustment instead of the denoising model 117 having the second adjustment. For example, during subsequent iterations of the adjustment, the molecular design engine 110 may further refine the parameters (e.g., weights, biases, etc.) of the denoising model 117 having the first adjustment before applying the further refined denoising model 117 to generate one or more additional updated molecules. In some cases, the denoising model 117 may be further adjusted to further increase the similarity (or decrease the mean squared error (MSE)) between the updated molecules generated by the denoising model 117 and the noisy three-dimensional representations (or their embedding representations) of sample molecules in the training dataset. In some cases, the molecular design engine 110 may continue to adjust the denoising model 117 until one or more criteria are met. For example, in some cases, the molecular design engine 110 may continue to adjust the parameters of the denoising model 117 (e.g., weights, biases, etc.) until the molecular design engine 110 has performed a threshold number of adjustment iterations. Alternatively and / or additionally, the molecular design engine 110 may continue to adjust the parameters of the denoising model 117 (e.g., weights, biases, etc.) until the similarity (e.g., mean squared error (MSE)) between the updated molecules generated by the denoising model 117 and the noisy three-dimensional representations (or their embedding representations) of sample molecules in the training dataset meets one or more thresholds.In some cases, the molecular design engine 110 may continue to adjust the parameters of the denoising model 117 (e.g., weights, biases, etc.) until the updated molecules generated by the denoising model 117 exhibit a threshold likelihood within the data distribution of molecules exhibiting one or more desired properties in the training dataset.

[0143] As will be explained in more detail below, if one or more criteria are met, the trained denoising model 117 may be applied to generate a three-dimensional representation (e.g., a voxelized representation) of the output molecule 162 by denoising at least the three-dimensional representation (e.g., a voxelized representation) of the input molecule 152. As shown in Figures 1A and 1B, the trained denoising model 117 may generate a three-dimensional representation (e.g., a voxelized representation) of the output molecule 162 by sampling at least from a noisy data distribution occupied by noisy three-dimensional representations (e.g., voxelized representations) of molecules exhibiting one or more desired properties (e.g., drug-like properties) based on the function 175, or alternatively, from a noisy latent distribution occupied by embedding representations of noisy three-dimensional representations (e.g., voxelized representations) of molecules. The sampling may include one or more iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling (e.g., Langevin Markov chain Monte Carlo), which can be induced by function 175 to include selecting one or more samples (or molecules) from an increasingly dense region of a noisy data distribution (or noisy latent distribution).

[0144] Figure 3B depicts a flowchart illustrating an example of process 325 for applying a molecular design computation model to generate a three-dimensional molecule in voxelized space, according to several exemplary embodiments. Referring to Figures 1A, 1B, 2, and 3B, process 325 may perform operation 206 of process 200 shown in Figure 2. In some cases, process 325 may be performed by a molecular design engine 110. For example, in some cases, the molecular design engine 110 may apply a molecular design computation model 115 (e.g., denoising model 117) to generate a three-dimensional representation of an output molecule (e.g., a voxelized representation) by denoising the three-dimensional representation of an input molecule (e.g., a voxelized representation). In some cases, the input molecule may be a random molecule (e.g., a molecule with a random selection of atomic types and / or positions) or a known molecule having one or more desired properties. Therefore, the three-dimensional representation of the input molecule (e.g., a voxelized representation) may contain noise that needs to be removed by the molecular design calculation model 115 so that the resulting three-dimensional representation of the output molecule (e.g., a voxelized representation) matches a molecule exhibiting one or more desired properties (e.g., drug-like properties). The molecular design calculation model 115 may remove noise from the three-dimensional representation of the input molecule (e.g., a voxelized representation) by sampling from a noisier data distribution (or a noisier latent distribution) that is more efficient to sample. This is because the smoother density transitions present therein allow for a proper exploration of the data distribution. Once the molecular design calculation model 115 generates a three-dimensional representation of the output molecule (e.g., a voxelized representation), as will be described in more detail below, the molecular design engine 110 may further generate one or more other representations of that output molecule, including, for example, a one-dimensional representation of the output molecule, a two-dimensional representation of the output molecule, and so on. The fact that an output molecule is generated by acting on a three-dimensional representation (e.g., a voxelized representation) of the input molecule, which captures the conformation (or three-dimensional structure) of the input molecule, means that the conformation (or three-dimensional) structure of the output molecule is more likely to match one or more desired properties (e.g., drug-like properties such as affinity, specificity, biological activity, development suitability, etc.).

[0145] In 332, the molecular design engine 110 may update the three-dimensional representation of the input molecule to generate an updated three-dimensional representation. In some exemplary embodiments, updating the three-dimensional representation may include the molecular design engine 110 applying the molecular design calculation model 115 to generate a three-dimensional representation (e.g., a voxelized representation) of the output molecule 162 by denoising the three-dimensional representation (e.g., a voxelized representation) of the input molecule 152. An example of this process is shown in Figure 1A, where the denoising engine 117 denoises the three-dimensional representation of the input molecule 152 to generate an updated three-dimensional representation 160. In some cases, the input molecule 152 may be a noise molecule (e.g., a molecule with randomly selected atomic types and / or positions) or a known molecule having one or more undesirable properties. This means that the three-dimensional representation of the input molecule 152 may contain at least some noise that would cause it to contradict that of a molecule exhibiting one or more desired properties (e.g., drug-like properties). Therefore, in some cases, the denoising engine 117 may be trained to update the three-dimensional representation of the input molecule 152 so that the resulting updated three-dimensional representation 160 matches the three-dimensional representation of a molecule exhibiting one or more desired properties.

[0146] In some exemplary embodiments, the molecular design calculation model 115 may apply a denoising model 117 to update the three-dimensional representation of the input molecule 152 based on a function 175. In some cases, function 175 may be a scoring function that outputs a value (e.g., a score) indicating the likelihood that each sample (or molecule) selected from the noisy data distribution is within the noisy data distribution. For example, in some cases, the value output by function 175 for a particular sample (or molecule) may indicate a local change in density at the location where the sample (or molecule) is selected. The denoising model 117 may update the three-dimensional representation of the input molecule 152 over several consecutive sampling iterations, based at least on the values ​​output by function 175. During each sampling iteration, the denoising model 117 may be applied to further update the three-dimensional representation of the input molecule 152 so that the resulting updated three-dimensional representation 160 is selected from a region of the noisy data distribution that is denser than those selected in one or more previous sampling iterations.

[0147] In some exemplary embodiments, the molecular design calculation model 115 may perform gradient-based Markov chain Monte Carlo (MCMC) sampling (e.g., Langevin Markov chain Monte Carlo (MCMC) sampling) of a noisy data distribution, in which the three-dimensional representation (e.g., voxelized representation) of the input molecule 152 is updated over multiple consecutive sampling iterations. In some cases, each iteration may include further updating of the three-dimensional representation of the input molecule 152 by the molecular design calculation model 115 so that it samples from increasingly dense regions of the noisy data distribution. Furthermore, in some cases, the updates performed on the three-dimensional representation of the input molecule 152 may accumulate over multiple consecutive iterations. For example, in some cases, the three-dimensional representation of the input molecule 152 may undergo a first update and a second update. The molecular design calculation model 115 may apply function 175 to determine a first value (e.g., a first score) of the three-dimensional representation of the input molecule 152 with a first update, and a second value (e.g., a second score) of the three-dimensional representation of the input molecule 152 with a second update. During subsequent iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling, the denoising model 117 may be applied to further update the three-dimensional representation of the input molecule 152 with a first update if the first and second values ​​indicate that the three-dimensional representation of the input molecule 152 with a first update is sampled from a denser region of the noisy data distribution and has a higher likelihood of being in the noisy data distribution than the three-dimensional representation of the input molecule 152 with a second update.

[0148] In some cases, one or more additional iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling may be performed, and the molecular design calculation model 115 applies a denoising model 117 to further modify the three-dimensional representation of the input molecule 152 until one or more criteria are met. For example, in some cases, the molecular design calculation model 115 may perform one or more additional iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling until a threshold number of sampling iterations have been performed. Alternatively and / or additionally, the molecular design calculation model 115 may perform one or more additional iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling until the function 175 outputs a value (e.g., a score) that satisfies one or more thresholds for the updated three-dimensional representation 160. The fact that a value (e.g., a score) associated with the updated three-dimensional representation 160 satisfies one or more thresholds may indicate that the updated three-dimensional representation 160 was selected from a region of a noisy data distribution with a sufficiently high density, and that the likelihood of the updated three-dimensional representation 160 being within the noisy data distribution satisfies one or more thresholds. In some cases, one or more criteria may also include the generation of threshold-level output molecules exhibiting one or more desired properties (e.g., at least one output molecule exhibits a threshold level for one or more drug-like properties such as affinity, specificity, biological activity, development suitability, etc.).

[0149] In step 336, the molecular design engine 110 may denoise the updated three-dimensional representation to generate a three-dimensional representation of the output molecule. In some exemplary embodiments of step 336, the molecular design calculation model 115 may denoise the three-dimensional representation of the input molecule 152 by sampling from a noisy data distribution occupied by noisy three-dimensional representations of molecules exhibiting one or more desired properties. As described above, the molecular design calculation model 115, including the denoising model 117, may be trained to approximate a noisy data distribution (instead of the true data distribution) by being trained to denoise a corrupted three-dimensional representation 182 of the sample molecule in order to reconstruct a noisy three-dimensional representation 184 of the sample molecule, rather than a clean three-dimensional representation of the sample molecule. Moreover, this noisy data distribution may exhibit smoother density transitions and is therefore more efficient to sample. The fact that the updated three-dimensional representation 160 is sampled from a noisy data distribution means that the updated three-dimensional representation 160 may undergo additional denoising. For example, Figure 1A shows that the molecular design engine 110 may apply a reconstruction model 118 to denoise the updated three-dimensional representation 160 and generate a three-dimensional representation (e.g., a voxelized representation) of the output molecule 162 from it. In some cases, the reconstruction model 118 may be trained to denoise the updated three-dimensional representation 160 in order to map the updated three-dimensional representation 160 back from a noisy data distribution to a true data distribution of molecules exhibiting one or more desired properties (e.g., drug-like properties). It should be understood that this denoising is different from the denoising that the denoising model 117 is trained to perform, which involves updating the three-dimensional representation of the input molecule 152 to sample from denser regions of the noisy data distribution that are more likely to be occupied by molecules exhibiting one or more desired properties.

[0150] In 338, the molecular design engine 110 may generate one or more other representations of the output molecule based on at least the three-dimensional representation of the output molecule. In some exemplary embodiments, the three-dimensional representation of the output molecule 162 (e.g., a voxelized representation), which is generated by the reconstruction model 118 denoising an updated three-dimensional representation 160 sampled from a noisy data distribution by the molecular computation model 115, may be further transformed into one or more other representations of the output molecule 162. For example, in some cases, the molecular design engine 110 may reconstruct the positions (e.g., coordinates) of atoms present in the output molecule 162 and one or more bonds between them, based on at least the three-dimensional representation (e.g., a voxelized representation) of the output molecule 162. In doing so, the molecular design engine 110 may determine other representations of the output molecule 162, including, for example, a one-dimensional representation of the output molecule 162 (e.g., a Simplified Molecular Input Line Notation (SMILES) string), a two-dimensional representation of the output molecule 162 (e.g., a molecular graph), and the like. In some cases, the molecular design engine 110 may reconstruct the positions of atoms present in the output molecule 162 by applying a peak detection technique that determines the positions (e.g., coordinates) of atoms based on one or more peaks in the atomic density contained in the three-dimensional representation (e.g., voxelized representation) of the output molecule 162, before determining one or more bonds based on the positions of atoms. Alternatively, the molecular design engine 110 may apply a machine learning model trained to convert the voxelized representation of the output molecule 162 into one or more other representations.

[0151] Figure 3C depicts a flowchart illustrating an example of process 350 for applying a molecular design computation model to generate a three-dimensional molecule in voxelized space, according to several exemplary embodiments. Referring to Figures 1-2 and 3C, process 350 may perform operation 206 of process 200 shown in Figure 2. In some cases, process 350 may be performed by a molecular design engine 110. For example, in some cases, the molecular design engine 110 may apply a molecular design computation model 115 (e.g., denoising model 117) to generate a three-dimensional representation of an output molecule, such as a voxelized representation of the output molecule, by denoising at least the three-dimensional representation of the input molecule (e.g., voxelized representation). In some cases, the three-dimensional representation of the input molecule may be denoised by updating at least the embedding representation of the three-dimensional representation of the input molecule (e.g., voxelized representation), rather than directly the three-dimensional representation of the input molecule, because at least the embedding representation may be more compact and more computationally efficient to operate. In some cases, the embedding representation of the three-dimensional representation of the input molecule may be generated by downsampling (or compressing) the three-dimensional representation of the input molecule, but it is also possible to generate the embedding representation without downsampling (or compressing) the three-dimensional representation of the input molecule at all. In the former case, the embedding representation of the three-dimensional representation of the input molecule may occupy a latent voxelization space, while in the latter case, the embedding representation of the three-dimensional representation of the input molecule may remain in the same discrete voxelization space as the original three-dimensional representation of the input molecule. It should be understood that the latent voxelization space may have lower dimensionality than the discrete voxelization space, and as a result, by operating on the embedding representation of the three-dimensional representation of the input molecule, the speed and computational efficiency of the generation process can be increased while achieving equivalent or better generation performance.

[0152] It should be understood that the molecular design calculation model 115 can denoise the embedding representation of the three-dimensional representation of the input molecule by sampling from a noisy latent distribution. That is, as described above, the molecular design calculation model 115 may be trained to approximate a noisy latent distribution rather than the true data distribution in order to avoid at least the steep density transitions present in the true data distribution. In other words, the updated embedding representation generated by the molecular design calculation model 115 when updating the embedding representation of the three-dimensional representation of the input molecule may still occupy a noisy latent distribution. This noisy latent distribution may be more efficient to sample because the smoother density transitions of the noisy latent distribution support a proper exploration of the data distribution. As will be described in more detail below, the updated embedding representation may undergo decoding and further denoising in order to "jump" back to the true data distribution. Furthermore, in some cases, the molecular design engine 110 may generate one or more other representations of the output molecule, including, for example, a one-dimensional representation of the output molecule, a two-dimensional representation of the output molecule, etc., based on the three-dimensional representation of the output molecule resulting from the decoding and denoising of the updated embedding representation. The fact that an output molecule is generated by acting on a three-dimensional representation (e.g., a voxelized representation) of the input molecule, which captures the conformation (or three-dimensional structure) of the input molecule, means that the conformation (or three-dimensional) structure of the output molecule is more likely to match one or more desired properties (e.g., drug-like properties such as affinity, specificity, biological activity, development suitability, etc.).

[0153] In 352, the molecular design engine 110 may encode a three-dimensional representation of the input molecule to generate an embedding representation of the input molecule. In some exemplary embodiments, the encoder 111 may encode a three-dimensional representation (e.g., a voxelized representation) of the input molecule 152 to generate an embedding representation 154 of the input molecule 152. An example of this is shown in Figure 1B. In the case of “seed generation”, the input molecule 152 may be a known molecule (e.g., a molecule from a validation set derived from a PubChem dataset, a QM9 molecular dataset, a Molecular Geometry Ensemble (GEOM) drug dataset, etc.). In some cases, the known molecule may exhibit one or more undesirable properties. When a known molecule is used as the input molecule 152, the generation process may be initialized with a voxel grid having an atomic density distribution corresponding to the types and positions of atoms expected to be found in the known molecule. Alternatively, the molecular design calculation model 115 may perform de novo generation, in which case the input molecule 152 may be a noise molecule whose atomic types and positions correspond to pure noise (e.g., uniform noise). When a noise molecule is used as the input molecule 152, the generation process may be initialized across the entire voxel grid without any prediction of atomic types and / or positions. In either case, the atomic types and / or positions in the input molecule 152 may not match those of a molecule exhibiting one or more desired properties (e.g., drug-like properties). Therefore, the molecular design calculation model 115 may update the three-dimensional representation of the input molecule 152 by updating at least the embedding representation 154, and apply to generate an updated embedding representation 156 so that the corresponding three-dimensional representation of the output molecule 162 can better match the three-dimensional representation of a molecule exhibiting one or more desired properties.

[0154] In some exemplary embodiments, the encoder 111 may encode the three-dimensional representation of the input molecule 152 (e.g., a voxelized representation) by downsampling or compressing at least the three-dimensional representation of the input molecule 152. Doing so may involve condensing at least some of the features present in the three-dimensional representation of the input molecule 152, thereby reducing the dimensionality (or amount of features) present in the three-dimensional representation of the input molecule 152. For example, the three-dimensional representation of the input molecule 152 contains 32,000 features (or atomic density values). If the TIFF2026518659000066.tif5170 voxel grid is included, the encoder 111 condenses at least some of its 32,000 features (or atomic density values) to include 64 features as an embedded representation 154 of the input molecule 152. You may also generate a voxel grid for TIFF2026518659000067.tif5170.

[0155] In some exemplary embodiments, the encoder 111 may generate an embedding representation 154 of the input molecule 152 with or without downsampling or compressing the three-dimensional representation (e.g., voxelized representation) of the input molecule 152. In some cases, the encoder 111 may implement an identity function, meaning that the embedding representation 154 may include features of the same quantities present in the three-dimensional representation of the input molecule 152 (e.g., atomic density values). Alternatively, if the embedding representation 154 is generated by downsampling the voxelized representation of the input molecule 152, the voxelized representation of the input molecule 152 may be projected from a higher-dimensional discrete voxelized space to a lower-dimensional latent space. The computational load imposed by sampling from a lower-dimensional latent space may be less than that imposed by direct sampling from a higher-dimensional discrete voxelized space. For example, if sampling from the discrete voxelized space is a resource-intensive task, such as when the input molecule 152 is large (e.g., containing 80-200 atoms) or when a large number of candidate molecules are generated from it, the molecular design engine 110 may sample from the latent voxelized space by applying the molecular design calculation model 115 to operate on the embedding representation 154 of the three-dimensional representation of the input molecule 152. It should be understood that sampling from the latent voxelized space may impose a moderate computational overhead, even when the input molecule 152 is large (e.g., containing 200 or more atoms) or when a large number of candidate molecules are generated.

[0156] In some exemplary embodiments, the encoder 111 may be part of an autoencoder (e.g., a variational autoencoder (VAE) such as a vector quantized variational autoencoder (VQ-VAE)) together with the decoder 119. In some cases, the encoder 111 may be trained to encode a voxelized representation of the input molecule 152 so that the decoder 119 can reconstruct a three-dimensional representation (e.g., a voxelized representation) of the input molecule 152 from the resulting embedding representation 154. To further illustrate, This shows the voxelized representation of the TIFF2026518659000068.tif5170 molecule. This refers to TIFF2026518659000069.tif6170 encoder 111, This refers to TIFF2026518659000070.tif6170 decoder 119, This represents TIFF2026518659000071.tif6170 embedded representation 154. It includes input molecule 152, etc. TIFF2026518659000072.tif5170 For example, according to formula (2) below To generate TIFF2026518659000073.tif6170 It may also be encoded using TIFF2026518659000074.tif6170.

number

[0157] According to equation (3), TIFF2026518659000076.tif6170 Nearest Neighbor Search In the learned shared codebook for TIFF2026518659000077.tif5170 By matching with one of the vectors in TIFF2026518659000078.tif5170 It may also be quantized to TIFF2026518659000079.tif5170.

number

[0158] TIFF2026518659000081.tif6170 According to formula (4) below, the original TIFF2026518659000082.tif5170 can be reconstructed.

number

[0159] In some cases, this operation may be nondifferentiable, and therefore there may be no defined gradient for the nearest neighbor search within the codebook of each latent embedding representation. Instead, the nearest neighbor search within the codebook is: Replace TIFF2026518659000088.tif6170 with one of the trained codebook embedding representations having the same dimensions. The gradient stopping (sg) operation is performed. The data entered in TIFF2026518659000089.tif6170 TIFF2026518659000090.tif6170 before quantization Output by TIFF2026518659000091.tif6170 The gradient may be copied to TIFF2026518659000092.tif6170. The gradient termination (sg) operation may also act as an identity function by copying without changing any variables in the forward direction. However, During a reverse pass updating the gradient of TIFF2026518659000093.tif6170, a gradient stop (sg) operation can prevent the gradient from flowing through gradient updates for the specific term to which the operation applies, since the gradient cannot be calculated for at least that term.

[0160] In some exemplary embodiments, an autoencoder (e.g., a variational autoencoder (VAE)) is formed. The training for TIFF2026518659000094.tif6170 is designed to reduce (or minimize) three distinct losses or loss terms. This may include adjusting TIFF2026518659000095.tif6170. The first loss term is: To generate TIFF2026518659000096.tif5170 Imported by TIFF2026518659000097.tif6170 Based on TIFF2026518659000098.tif5170 Generated by TIFF2026518659000099.tif6170 The reconstruction loss corresponding to the difference between TIFF2026518659000100.tif5170 may be included (e.g., mean squared error (MSE) reconstruction loss). The second loss term is: Output by TIFF2026518659000101.tif6170 By moving toward TIFF2026518659000102.tif6170, it is used to quantize the latent space. You may run a study of the codebook for TIFF2026518659000103.tif5170. The third loss term is: A commitment loss may be quantified to commit to TIFF2026518659000104.tif6170 and ensure that its output does not increase arbitrarily. This third loss term is, It may be associated with TIFF2026518659000105.tif6170, and the commitment cost weights may also be hyperparameters set through experimentation. Equation (5) below is, To train TIFF2026518659000106.tif6170 This is an example of TIFF2026518659000107.tif5170.

number

[0161] In 354, the molecular design engine 110 may generate an updated embedding representation by updating at least the embedding representation of the three-dimensional representation of the input molecule. In some exemplary embodiments, the molecular design engine 110 may apply a molecular design calculation model 115 (e.g., denoising model 117) to denoise the three-dimensional representation of the input molecule 152 by updating at least the embedding representation 154 of the three-dimensional representation of the input molecule 152 and generating an updated embedding representation 156. For example, in some cases, the three-dimensional representation of the input molecule 152 may contain noise that contributes to a discrepancy between the types and / or locations of atoms present in the input molecule 152 and the types and / or locations of atoms in a molecule that exhibit one or more desired properties (e.g., drug-like properties). In other words, the molecular design calculation model 115 may update the embedding representation 154 of the three-dimensional representation of the input molecule 152 to increase the likelihood that the resulting output molecule 162 exhibits one or more desired properties. As described above, the noise removed from the embedding representation 154 by the denoising model 117 should not be confused with the noise that projects the three-dimensional representation of the input molecule 152 from its true data distribution, which exhibits sawtooth density transitions, to a noisy data distribution exhibiting smoother density transitions, for more efficient sampling (e.g., gradient-based Markov chain Monte Carlo (MCMC) sampling). As will be described in more detail below, by updating the embedding representation 154 of the three-dimensional representation of the input molecule 152, the molecular design calculation model 115 (e.g., the denoising model 117) may traverse the smoother density of the noisy data distribution to sample the updated embedding representation 156 from the region of the noisy data distribution that exhibits an increasing likelihood of being within the noisy data distribution before "jumping" back to the true data distribution when a sample exhibiting a threshold likelihood of being within the noisy data distribution is selected.

[0162] In some exemplary embodiments, the denoising model 117 may apply updates to the embedding representation 154 of the three-dimensional representation of the input molecule 152 that correspond to changes in the types and / or positions of atoms present in the input molecule 152. If the encoder 111 implements an identity function and the embedding representation 154 is generated without any downsampling (or compression) of the underlying three-dimensional representation (e.g., voxelized representation) of the input molecule 152, the denoising model 117 may update the embedding representation 154 by updating the atomic density of one or more voxels in at least one channel of the embedding representation 154. Alternatively, if the generation of the embedding representation 154 involves downsampling (or compression) of the underlying three-dimensional representation (e.g., voxelized representation) of the input molecule 152, the denoising model 117 may update the embedding representation 154 by updating one or more values ​​present in at least the embedding representation 154, at least a portion of which condense multiple atomic density values ​​contained within the three-dimensional representation (e.g., voxelized representation) of the input molecule 152.

[0163] In some exemplary embodiments, the molecular design calculation model 115 may apply a denoising model 117 to update the embedding representation 154 of the input molecule 152 based on a function 175. In some cases, the function 175 may output a value (e.g., a score) indicating the likelihood that each sample (or molecule) selected from the noisy data distribution is within the noisy data distribution. For example, in some cases, the value output by the function 175 for a particular sample (or molecule) may indicate a local change in density at the location where the sample (or molecule) is selected. The denoising model 117 may update the embedding 154 over several consecutive sampling iterations based on at least the value output by the function 175. During each sampling iteration, the denoising model 117 may be applied to further update the embedding representation 154 so that the resulting updated embedding representation 156 is selected from a region of the noisy data distribution that is denser than that selected in the previous sampling iteration.

[0164] In some exemplary embodiments, the molecular design calculation model 115 may perform gradient-based Markov chain Monte Carlo (MCMC) sampling (e.g., Langevin Markov chain Monte Carlo (MCMC) sampling) of a noisy data distribution, in which the embedding representation 154 of the three-dimensional representation (e.g., voxelized representation) of the input molecule 152 is updated over multiple consecutive sampling iterations, with each iteration sampling from a region of the noisy data distribution with increasing density to increase the likelihood that the resulting updated embedding representation 156 is in the noisy data distribution. Furthermore, in some cases, the updates performed on the embedding representation 154 of the input sequence 152 may be cumulative over multiple consecutive iterations. To further illustrate, consider an example in which the embedding representation 154 of the three-dimensional representation of the input molecule 152 undergoes a first update and a second update. The molecular design calculation model 115 may apply function 175 to determine a first value (e.g., a first score) for the embedding representation 154 with a first update, and a second value (e.g., a second score) for the embedding representation 154 with a second update. During subsequent iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling, the denoising model 117 may be applied to further update the embedding representation 154 of the input molecule 154 with a first update if the first and second values ​​indicate that the embedding representation 154 with a first update is sampled from a denser region of the noisier data distribution and exhibits a higher likelihood of being in a noisier data distribution than the embedding representation 154 with a second update. In some cases, one or more additional iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling may be performed, and the molecular design calculation model 115 applies a denoising model 117 to further modify the embedding representation 154 of the three-dimensional representation (e.g., voxelized representation) of the input molecule 152 until one or more criteria are met. For example, in some cases, the molecular design calculation model 115 may perform one or more additional iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling until a threshold number of sampling iterations have been performed.Alternatively and / or additionally, the molecular design calculation model 115 may perform one or more additional iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling for the updated embedding representation 156 until the function 175 outputs a value (e.g., a score) that satisfies one or more thresholds. The fact that a value (e.g., a score) associated with the updated embedding representation 156 satisfies one or more thresholds may indicate that the updated embedding representation 156 was selected from a region of a noisy data distribution with a sufficiently high density, and that the likelihood of the updated embedding representation 156 being within the noisy data distribution satisfies one or more thresholds. In some cases, one or more criteria may also include the generation of threshold output molecules exhibiting one or more desired properties (e.g., at least one output molecule exhibits a threshold level for one or more drug-like properties such as affinity, specificity, biological activity, development suitability, etc.).

[0165] In 356, the molecular design calculation model 115 may decode the updated embedding representation to generate a noisy three-dimensional representation of the output molecule. In some exemplary embodiments, the molecular design engine 110 may update the embedding representation 154 of the three-dimensional representation of the input molecule 152, and when the molecular design calculation model 115 (e.g., denoising model 117) is applied to generate the updated embedding representation 156, the decoder 119 may be applied to decode the updated embedding representation 156 to generate a noisy three-dimensional representation 158 of the output molecule 162. Decoding the updated embedding representation 156 may map the updated embedding representation 156 from a latent voxelized space occupied by the embedding representations of the three-dimensional representations of various molecules to a latent discrete space. However, as will be explained in more detail below, the latent discrete space may be a noisy latent space, which means that the noisy three-dimensional representation 158 generated by the decoder 119 that decodes the updated embedding representation 156 may require further denoising in order to project the noisy three-dimensional representation 158 back to the true data distribution of molecules exhibiting one or more desired properties.

[0166] In some exemplary embodiments, the decoder 119 of the molecular design engine 110 may generate a noisy three-dimensional representation 158 by decoding at least an updated embedding representation 156 generated by the molecular design computation model 115 (e.g., the denoising engine 117). As described above, in some cases, the decoder 119 may, together with the encoder 111, form part of an autoencoder (e.g., a variational autoencoder such as a vector quantized variational autoencoder (VQ-VAE)). In some cases, the encoder 111 and the decoder 119 may be trained in parallel, and the encoder 111 may be trained to generate an embedding representation of the three-dimensional representation of the molecule (e.g., a voxelized representation), such as an embedding representation 154 of the three-dimensional representation of the input molecule 152, from which the decoder 119 can reconstruct the original three-dimensional representation (e.g., a voxelized representation). Thus, once the updated embedding representation 156 is generated, the decoder 119 may apply thereto to reconstruct a noisy three-dimensional representation 158 of the output molecule 162.

[0167] In some cases, decoding the updated embedding representation 156 may include upsampling (or decompressing) the updated embedding representation 156, which may project the updated embedding representation 156 from the latent voxelization space back into the discrete voxelization space. The noisy three-dimensional representation 158 (e.g., a noisy voxelized representation) of the output molecule 162 may have the same dimensionality (or number of features) as the three-dimensional representation (e.g., a voxelized representation) of the input molecule 152 taken up by the molecular design engine 110 in operation 352. For example, in some cases, the three-dimensional representation of the input molecule 162 is TIFF2026518659000109.tif5170 may include a voxel grid, meaning that the three-dimensional representation of the input molecule 152 may contain 32,000 features (or atomic density values). On the other hand, each of the embedding representation 154 on which the molecular design calculation model 115 operates and the resulting updated embedding representation 156 has 64 features. TIFF2026518659000110.tif5170 may include a voxel grid. In some cases, the decoder 119 may provide a noisy three-dimensional representation 158 (e.g., a noisy voxelized representation) of the output molecule 162. To generate the TIFF2026518659000111.tif5170 voxel grid, it contains The updated embedding representation 156 may be decoded by upsampling (or decompressing) the voxel grid of TIFF2026518659000112.tif5170. It should be understood that this upsampling (or decompression) may recover 32,000 features (or atomic density values) within the noisy three-dimensional representation 158 (e.g., the voxelized representation) of the output molecule 162. As described above, these 32,000 features (or atomic density values) may indicate the positions of various atoms present in the output molecule 162. Furthermore, the 32,000 features (or atomic density values) may span one or more channels, each corresponding to a type of atom that may be present in the output molecule 162.

[0168] In 358, the molecular design engine 110 may denoise a noisy three-dimensional representation of the output molecule to generate a three-dimensional representation of the output molecule. In some exemplary embodiments, the denoising engine 117 of the molecular design calculation model 115 may generate a three-dimensional representation (e.g., a voxelized representation) of the output molecule 162 by denoising a noisy three-dimensional representation 158 generated by decoding the updated embedding representation 156 by the decoder 119. As described above, in some cases, the molecular design calculation model 115 (e.g., the denoising model 117) may generate the updated embedding representation 156 over one or more iterations of a gradient-based Markov chain Monte Carlo method (e.g., Langevin Markov chain Monte Carlo method). In doing so, the molecular design calculation model 115 may traverse the noisy latent distribution, at least based on the output of function 175 (e.g., the score output by function 175), to sample the updated embedding representation 156 from a denser region of the noisy latent distribution occupied by embedding representations of three-dimensional representations of molecules that are more likely to exhibit one or more desired properties (e.g., drug-like properties). However, decoding the updated embedding representation 156 only maps the updated embedding representation 156 from the latent voxelization space to the discrete voxelization space, and the noisy three-dimensional representation 158 still occupies the noisy data distribution, not the true data distribution of molecules exhibiting one or more desired properties. Therefore, in some cases, the reconstruction model 118 may be applied to map the noisy three-dimensional representation 158 from the noisy data distribution to the true data distribution. In some cases, this may constitute a “jump” back to the true data distribution, meaning that the three-dimensional representation of the output molecules 162 generated therefrom occupies the true data distribution.

[0169] In some cases, the reconstruction model 118 may share the same architecture (e.g., an artificial neural network (ANN)) as the denoising model 117, which is trained to denoise the embedding representation 154 of the three-dimensional representation of the input molecule 152 and traverse a noisy latent distribution to generate an updated embedding representation 156. However, as mentioned above, the reconstruction model 118 may be trained to denoise different types of noise. Therefore, in some cases, the reconstruction engine 118 may be trained based on the training dataset to denoise the noisy three-dimensional representation 182 of the sample molecule and reconstruct the original three-dimensional representation 182 from there. In contrast, the denoising engine 117 may be trained to reconstruct the embedding representation 186 of the noisy three-dimensional representation 182 of the sample molecule from the corrupted embedding representation 188. In this regard, training the denoising engine 117 may include tuning one or more parameters of the denoising engine 117 (e.g., an artificial neural network (ANN)) to reduce (or minimize) the difference (e.g., mean squared error (MSE)) between the original three-dimensional representation of the sample molecule and the three-dimensional representation of the sample molecule that the denoising engine 117 reconstructs from the noisy three-dimensional representation of the sample molecule.

[0170] For further illustration, see Operation 252. Let's consider TIFF2026518659000113.tif6170. As mentioned above, It may be generated by encoding TIFF2026518659000114.tif6170. In some cases, TIFF2026518659000115.tif6170 (for example, Gaussian noise such as isotropic Gaussian noise) may be added. For example, in some cases, It has an unscaled unit covariance matrix at TIFF2026518659000116.tif5170. TIFF2026518659000117.tif5170 (for example, Gaussian noise such as isotropic Gaussian noise) may be added according to equation (6) below.

[0171]

number

[0172] The noise reduction engine 117, which may be represented as TIFF2026518659000119.tif6170, Before (or before) TIFF2026518659000120.tif5170 was added TIFF2026518659000121.tif6170 Denoised by denoising engine 117 While reducing (or minimizing) the reconstruction loss (e.g., mean squared error (MSE) reconstruction loss) between TIFF2026518659000122.tif6170, TIFF2026518659000123.tif6170 may be trained to denoise and restore it. To generate TIFF2026518659000124.tif6170 The noise reduction performed by TIFF2026518659000125.tif6170 is shown in equation (7) below. On the other hand, equation (8) is, To train TIFF2026518659000126.tif6170 This shows TIFF2026518659000127.tif5170, which is, Before (or before) TIFF2026518659000128.tif5170 was added This includes reducing (or minimizing) the difference (e.g., mean squared error (MSE)) between TIFF2026518659000129.tif6170 and the original file.

number

number

[0173] In 360, the molecular design engine 110 may generate one or more other representations of the output molecule based on at least a three-dimensional representation of the output molecule. In some exemplary embodiments, the molecular design engine 110 may generate one or more other representations of the output molecule 162 based on at least a voxelized representation of the output molecule 162, including, for example, a one-dimensional representation of the output molecule 162 (e.g., a Simplified Molecular Input Line Notation (SMILES) string), a two-dimensional representation of the output molecule 162 (e.g., a molecular graph), and so on. For example, in some cases, the molecular design calculation model 110 may reconstruct the positions (e.g., coordinates) of atoms present in the output molecule 162 and the bonds between them from at least a voxelized representation of the output molecule 162. In some cases, the molecular design engine 110 may apply a peak detection technique to determine the positions (e.g., coordinates) of atoms present in the output molecule 162 based on one or more peaks of atomic density included in the voxelized representation of the output molecule 162, before determining one or more bonds based on the positions of the atoms. Alternatively, the molecular design engine 110 may apply a machine learning model trained to convert the voxelized representations of the output molecules 162 into one or more other representations.

[0174] As described above, in some exemplary embodiments, the molecular design calculation model 115, including the denoising model 117, may operate on a three-dimensional representation of the molecule instead of a one-dimensional or two-dimensional representation of the molecule. This is because, at the very least, realistic and reasonable molecules exhibiting specific desired properties are likely to be generated based on a representation of the molecule that captures the molecular composition (e.g., constituent atoms) and conformation (or three-dimensional structure). In some cases, the molecular design calculation model 115, including the denoising model 117, may operate on a voxelized representation of the molecule. Unlike conventional three-dimensional representations of molecules (e.g., point cloud representations), a voxelized representation of a molecule can represent both atomic type and position as one or more continuous (e.g., Gaussian) distributions across a voxel grid centered on the atomic coordinates of individual atoms. Therefore, unlike conventional three-dimensional representations of molecules (e.g., point cloud representations), the molecular design calculation model 115 can apply a denoising model 117 to work on a voxelized representation of an input molecule without requiring any workarounds to harmonize various types of data distributions (e.g., discrete distributions of atomic types and continuous distributions of atomic positions), nor without any prior knowledge of the number of atoms present in the resulting output molecule.

[0175] For further illustration, Figure 4 depicts examples of voxelized representations and corresponding two-dimensional representations of different molecules according to several exemplary embodiments. For example, Figure 4 shows a voxelized representation 400 and a two-dimensional representation 450 of a molecule. In some exemplary embodiments, the voxelized representation 400 of a molecule may be generated by partitioning (or discretizing) the three-dimensional space around the constituent atoms into a voxel grid 410, where each type of atom (or element) present in the molecule is represented by a different grid channel. This partitioning (or discretization) is performed It is possible to generate TIFF2026518659000132.tif7170, TIFF2026518659000133.tif5170 shows the length of each grid side. This shows the number of channels (e.g., the amount of different types of atoms (or elements)) in the TIFF2026518659000134.tif5170 dataset.

[0176] In some cases, the voxel grid 410 may be a three-dimensional grid of voxels organized into continuous layers of rows and columns. Each voxel in the voxel grid 410 may be a volume element, such as a three-dimensional cube, formed at the intersection of rows and columns. Furthermore, each voxel in the voxel grid 410 may be associated with a value indicating the atomic density at the corresponding position (for example, (Having TIFF2026518659000135.tif5170). For a single molecule, the corresponding voxelized representation may be a box around the center of the molecule which is then divided into voxels. To generate the voxelized representation 400 of a molecule, each constituent atom may be converted to a three-dimensional continuous (e.g., Gaussian) density according to formula (9) below. For example, the example of the voxel grid 410 shown in Figure 4 may include a first atomic density 415a representing a first atom of a first type and a second atomic density 415b representing a second atom of a second type.

number

number

[0177] As described above, in some cases, the atomic density within the voxelized representation 400 of the molecule may be centered on the atoms present in the molecule. Therefore, TIFF2026518659000144.tif7170 The maximum value (e.g., a value of 1) may be taken at the center of the atom and decrease to a minimum value (e.g., a value of 0) as the distance from the center of the atom increases. All channels within the voxel grid may be independent; that is, the channels do not interact or share a volume contribution. In some cases, the size of the voxel grid 410 contained in the voxelized representation 400 of the molecule may correspond to the size of the molecule being represented (e.g., the number of constituent atoms). For example, in some cases, the voxel grid 410 may be a [32×32×32] voxel grid if the molecule has fewer atoms (e.g., the QM9 molecular dataset) and a [64×64×64] voxel grid if the molecule has more atoms (e.g., the Molecular Geometry Ensemble (GEOM) drug dataset). Furthermore, in some cases, the number of channels in the voxelized representation 400 of the molecule may correspond to the number of atomic types (or elements) present in the molecule. For example, the voxelized representation of molecules in the QM9 molecular dataset may include five channels for the five types of atoms that make up those molecules (e.g., carbon (C), hydrogen (H), oxygen (O), nitrogen (N), and fluorine (F)). On the other hand, the voxelized representation of molecules in the Molecular Geometry Ensemble (GEOM) drug dataset may include eight channels for the eight types of atoms present in those molecules (e.g., carbon (C), hydrogen (H), oxygen (O), nitrogen (N), fluorine (F), sulfur (S), chlorine (Cl), and bromine (Br)). Therefore, the voxelized representation of each molecule in the QM9 molecular dataset may include: TIFF2026518659000145.tif6170 may also be included, and the voxelized representation of each molecule in the Molecular Geometry Ensemble (GEOM) drug dataset is, It may also include TIFF2026518659000146.tif6170.

[0178] As described above, in some exemplary embodiments, the molecular design calculation model 115, including the denoising model 117, may be trained to approximate and then sample from a noisy data distribution of a noisy voxelated representation of a molecule, or possibly a noisy embedding representation of a voxelated representation of a molecule, instead of the true data distribution of a voxelated representation of a molecule that is not disturbed by any noise. Training the denoising model 117 to approximate a noisy data distribution of a molecule, such as a noisy data distribution of a noisy voxelated representation of a molecule exhibiting a particular desired property (e.g., drug-like property), or its noisy embedding representation, may involve determining function 175 such that for each voxelated representation of a molecule (or its noisy embedding representation) sampled from the noisy data distribution, function 175 outputs a value indicating the density of the corresponding location in the noisy data distribution. If function 175 is a scoring function, function 175 may output a score corresponding to a local change (or gradient) in the density of the noisy data distribution. Therefore, if function 175 is a scoring function, the score output by function 175 for a noisy voxelized representation (or noisy embedding representation) of the numerator may indicate a local change in density at the corresponding location in the noisy data distribution.

[0179] In some cases, the denoising engine 117 may be trained to denoise a noisy voxelized representation of a molecule, or possibly a noisy embedding representation of a voxelized representation of a molecule, generated by the molecular design calculation model 115 (e.g., the denoising model 117). For further illustration, Figure 5A depicts a schematic diagram illustrating an example of training the denoising engine 117 to denoise a noisy voxelized representation of a molecule according to several exemplary embodiments. As shown in Figure 5A, the training dataset for training the denoising engine 117 may be generated to contain multiple training samples, each corresponding to a sample molecule. For example, Figure 5A shows a sample molecule 500, which may be a known molecule from the PubChem dataset, the QM9 molecular dataset, the Molecular Geometry Ensemble (GEOM) drug dataset, etc. The sample molecule 500 may be rendered in a one-dimensional representation (e.g., a Simplified Molecular Input Line Notation (SMILES) string) or a two-dimensional representation (e.g., a molecular graph), neither of which adequately captures the conformation (or three-dimensional structure) of the sample molecule 500. Therefore, in some cases, a one-dimensional or two-dimensional representation of sample molecule 500 may be converted to a three-dimensional representation of sample molecule 500 in order to generate training samples to be included in the training dataset. For example, in some cases, a one-dimensional or two-dimensional representation of sample molecule 500 is shown in Figure 5A. It may be converted to TIFF2026518659000147.tif5170. (500 sample molecules) The type and position of atoms present in the sample molecule 500 can be represented together as one of many continuous (e.g., Gaussian) densities across a voxel grid, centered on individual atoms present in the sample molecule 500.

[0180] Referring again to Figure 5A, in some cases, 500 sample molecules TIFF2026518659000149.tif5170 Noisy To generate TIFF2026518659000150.tif5170, TIFF2026518659000151.tif5170 (for example, Gaussian noise such as isotropic Gaussian noise) may be included. The addition of TIFF2026518659000152.tif5170 is true, occupied by the clean (or original) voxelized representation of the molecule. TIFF2026518659000153.tif6170 Noisy Voxelized Representation of the Molecules TIFF2026518659000154.tif6170 can be projected. As mentioned above, molecular design calculation model 115 of 500 sample molecules If you want to work directly on the clean (or original) voxelized representation of the molecule from TIFF2026518659000155.tif6170, The sawtooth energy landscape of TIFF2026518659000156.tif6170 is used when sampling from molecular design calculation model 115. This may prevent proper exploration of TIFF2026518659000157.tif6170. In contrast, TIFF2026518659000158.tif6170 may exhibit a smoother energy landscape with gradual gradient changes, which is because molecular design calculation model 115 is noisy. Sampling from TIFF2026518659000159.tif6170 means that greater diversity may be obtained in the resulting output molecules. Therefore, in some cases, the denoising engine 117 may be noisy, and the molecular design calculation model 115 may be noisy. The noisy voxelized representation of molecules generated by sampling from TIFF2026518659000160.tif6170 can be applied to denoise the noisy voxelized representation of molecules. TIFF2026518659000161.tif5170 may be trained to denoise noisy voxelized representations of molecules. In some cases, as described in more detail below, 500 sample molecules TIFF2026518659000162.tif5170 may have undergone downsampling (or compression) before being added, which means that the denoising engine 117 can be trained to denoise the noisy embedding representation of the molecule's voxelized representation instead of the noisy voxelized representation of the molecule shown in Figure 5A.

[0181] Referring again to Figure 5A, the noise reduction engine 117 is noisy TIFF2026518659000163.tif5170 may be trained to denoise. In some cases, the denoising engine 117 may be trained to denoise at least the corresponding clean By restoring TIFF2026518659000164.tif5170 from there, the noisy TIFF2026518659000165.tif5170 may be trained to denoise. For example, in some cases, the denoising engine 117 may be trained to denoise noisy It may also be an encoder-decoder three-dimensional convolutional neural network (CNN) trained to map noisy voxels in TIFF2026518659000166.tif5170 to corresponding clean voxels. In doing so, the denoising engine 117 Denoised approximation of TIFF2026518659000167.tif5170 TIFF2026518659000168.tif5170 may be generated. For example, in some cases, the denoising engine 117 is trained to denoise the TIFF2026518659000169.tif5170 and its corresponding clean This may include adjusting the parameters of the denoising engine 117 to reduce (or minimize) the difference (e.g., mean squared error (MSE)) between TIFF2026518659000170.tif5170 and the original file. In some cases, TIFF2026518659000171.tif5170 may be set as a hyperparameter of the denoising engine 117. Furthermore, in some cases, during training of the denoising engine 117 TIFF2026518659000172.tif5170 can be maintained at a fixed (or constant) level, thereby reducing the complexity of the training process compared to diffusion models. Unlike natural images, it contains more structural information than texture information on the 500 sample molecules. Due to the nature of TIFF2026518659000173.tif5170, single-step denoising (in contrast to spreading over multiple time steps) Please understand that it may be sufficient to reconstruct TIFF2026518659000174.tif5170.

[0182] In some exemplary embodiments, the molecular design calculation model 115 may apply a denoising model 117 to generate a voxelized representation of an output molecule by denoising a noisy voxelized representation of an input molecule over at least one or more iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling (e.g., Langevin Markov chain Monte Carlo (MCMC) sampling). In some cases, the denoising model 117 may apply You can also sample from TIFF2026518659000175.tif6170, which is, TIFF2026518659000176.tif5170Occupied by molecules exhibiting one or more desired properties (e.g., drug-like properties), This includes traversing towards increasingly dense regions in TIFF2026518659000177.tif6170. For further illustration, Figure 5B shows: Traversing across TIFF2026518659000178.tif5170 is At TIFF2026518659000179.tif5170 Select TIFF2026518659000180.tif5170, At TIFF2026518659000181.tif5170 Select TIFF2026518659000182.tif5170, At TIFF2026518659000183.tif5170 This indicates that you will select TIFF2026518659000184.tif5170. In some cases, Traversing the data distribution with a high density of TIFF2026518659000185.tif5170 is: TIFF2026518659000186.tif5170 Noisy It was sampled from a higher density region than TIFF2026518659000187.tif6170, while, Function 175 may be induced to sample from a region with an even higher density than TIFF2026518659000188.tif6170. In some cases, each iteration of the gradient-based Markov chain Monte Carlo (MCMC) method may include further modification of the sample (or molecule) selected during the previous iteration. Thus, as shown below, Selected from TIFF2026518659000189.tif6170 It may be generated based on TIFF2026518659000190.tif6170. The following formula (10) is: This represents the traverse of TIFF2026518659000191.tif6170.

number

[0183] Referring again to Figure 5B, in some cases the noise reduction engine 117 is noisy. Selected from TIFF2026518659000197.tif6170, corresponding noisy When denoising TIFF2026518659000198.tif5170, TIFF2026518659000199.tif5170 may be generated. As mentioned above, it is noisy. Noise reduction for TIFF2026518659000200.tif5170 can be done, for example, By applying TIFF2026518659000201.tif6170, It can be projected back to TIFF2026518659000202.tif6170. This constitutes the "jump" shown in Figure 5B. Furthermore, in the example shown in Figure 5B, While TIFF2026518659000203.tif6170 is being traversed and denoising model 117 is being applied to select samples from it, A "jump" back to TIFF2026518659000204.tif6170 may be performed in each sampling iteration. For example, Selected from TIFF2026518659000205.tif6170 It may be generated when projected back to TIFF2026518659000206.tif6170, on the other hand, TIFF2026518659000207.tif5170 (followed by) TIFF2026518659000208.tif5170 has a lot of noise. TIFF2026518659000209.tif6170 has been denoised. This can be generated when projected back to TIFF2026518659000210.tif6170.

[0184] [Table 1]

[0185] In some exemplary embodiments, the noise reduction model 117 continues to reduce noise until one or more criteria are met. You can also traverse TIFF2026518659000212.tif6170 and select samples from there. For example, denoising model 117 will perform a threshold number of sampling iterations at that point. TIFF2026518659000213.tif5170 It may continue to "traverse" the energy landscape of noisy data distributions. Alternatively and / or additionally, denoising model 117, If it exhibits the threshold likelihood found in TIFF2026518659000214.tif6170, Until TIFF2026518659000215.tif5170 is selected, You may continue traversing the energy landscape of TIFF2026518659000216.tif6170. For further illustration, Figure 5C shows that the denoising model 117 includes, for example, samples 510a to 510f. This demonstrates that it is applied to select multiple consecutive samples from TIFF2026518659000217.tif6170. In the example shown in Figure 5C, sampling (e.g., gradient-based Markov chain Monte Carlo (MCMC) sampling) is performed by molecular design calculation model 115. From TIFF2026518659000218.tif6170 It may be started by applying the noise removal model 117 to select TIFF2026518659000219.tif5170. In some cases, TIFF2026518659000220.tif5170 noise removal model 117 may include updating the voxelized representation (or its noisy embedded representation) of the corresponding molecule with noise.

[0186] As shown in Figure 5C, TIFF2026518659000221.tif5170 may be denoised to generate. This denoising operation TIFF2026518659000222.tif6170 may constitute a "jump" back. Each subsequent sampling iteration may include applying the noise removal model 117 to further update the voxelized representation with noise of the molecule selected during the previous sampling iteration. In the example shown in Figure 5C, the molecular design calculation model 115 TIFF2026518659000223.tif6170 may continue to apply the noise removal model 117 until consecutive samples of are selected. TIFF2026518659000224.tif5170 may be denoised, for example, by the noise removal engine 117 to generate. By doing so, TIFF2026518659000225.tif6170 can be projected back. TIFF2026518659000226.tif5170 number of sampling iterations and TIFF2026518659000227.tif5170 It should be understood that the amount of samples selected from can be determined. TIFF2026518659000228.tif5170 increases, the updates performed on the initial input molecule (e.g., "seed" molecule) may increase. TIFF2026518659000229.tif5170 The difference between the initial input molecule (e.g., "seed" molecule) and the final output molecule, and the novelty of the final output molecule can be increased.

[0187] In some exemplary embodiments, TIFF2026518659000230.tif6170 is selected, TIFF2026518659000231.tif5170 is denoised to the corresponding when TIFF2026518659000232.tif5170 is generated, the molecular design engine 110 may generate one or more other representations based on TIFF2026518659000233.tif5170. For example, in some cases, the molecular design engine 110 may, at least generate a one-dimensional representation (e.g., a Simplified Molecular-Input Line-Entry System (SMILES) string) and / or a two-dimensional representation (e.g., a molecular graph) of the corresponding molecule based on TIFF2026518659000234.tif5170.

[0188] FIG. 5D depicts a schematic diagram showing an example of a process for generating other molecular representations from TIFF2026518659000235.tif5170 according to some exemplary embodiments. In the example shown in FIG. 5D, the molecular design engine 110 may, at least determine the atoms present in the corresponding molecule by identifying the peaks of TIFF2026518659000236.tif5170 (e.g., atom density values that meet one or more thresholds). Further, the molecular design engine 110 may determine one or more bonds that interconnect the atoms present in the molecule. A one-dimensional or two-dimensional representation of the molecule may be generated based at least on the atoms and bonds. Alternatively, in some cases, the molecular design engine 110 may apply a machine learning model trained to convert TIFF2026518659000237.tif5170 to one or more other representations of the corresponding molecule.

[0189] In some exemplary embodiments, instead of operating in a noisy discrete voxelized space, as shown in Figures 5A to 5D, for example, the molecular design calculation model 115 may operate in a noisy latent voxelized space. For example, in some cases the molecular design calculation model 115 may operate in a noisy latent voxelized space. Instead of applying denoising model 117 to denoise TIFF2026518659000238.tif5170, denoising model 117 is: The noisy embedded representation of TIFF2026518659000239.tif5170 may be applied to denoise it. For further illustration, Figure 6 depicts a schematic diagram illustrating an example of the process by which the molecular design calculation model 115 generates a voxelized representation of a molecule by operating in a noisy latent voxelized space, according to several exemplary embodiments. Referring to Figure 6, an input molecule 600 that can be rendered in a one-dimensional or two-dimensional representation may be converted to a three-dimensional representation of the input molecule 152. In some cases, the three-dimensional representation of the input molecule 152 may be a voxelized representation of the input molecule 152, which together represents the types and positions of atoms in the input molecule 152 as one or more continuous distributions of atomic density across a voxel grid. In some cases, instead of applying the denoising model 117 to operate directly on the noisy voxelized representation of the input molecule 152, the encoder 111 may, Before being added to the embedding representation 154 of the voxelized representation of the input molecule 152, the embedding representation 154 of the voxelized representation of the input molecule 152 may be generated first. The resulting noisy embedding representation 156 may undergo one or more iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling (e.g., Langevin Markov chain Monte Carlo (MCMC) sampling). For example, each iteration of gradient-based Markov chain Monte Carlo (MCMC) sampling may include the molecular design calculation model 115 applying a denoising model 117 to denoise the noisy embedding representation 156 by updating the noisy embedding representation 156. As described above, updating the noisy embedding representation 156 in this way may be equivalent to selecting one or more samples from a noisy data distribution occupied by noisy embedding representations of voxelized representations of molecules exhibiting one or more desired properties. Sampling may be induced by a function (e.g., a score function) such that consecutive samples are selected from increasingly dense regions of a noisy data distribution, and the noisy data distribution is more likely to be occupied by noisy embedding representations of voxelized representations of molecules exhibiting one or more desired properties.

[0190] Referring again to Figure 6, the molecular design calculation model 115 may generate an updated embedding representation 156 by updating the embedding representation 154 of the voxelized representation of the input molecule 152 over at least one or more iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling. As shown in Figure 6, the embedding representation 154 may be denoised by, for example, a denoising engine 117, thereby generating the updated embedding representation 156. Denoising of the embedding representation 154 may include sampling the updated embedding representation 156 from a noisy latent distribution of molecules exhibiting one or more desired characteristics. Furthermore, as shown in Figure 6, the decoder 119 may decode the updated embedding representation 156 to generate the corresponding voxelized representation of the output molecule 162. Decoding the updated embedding representation 156 may involve projecting the updated embedding representation 156 back from the latent voxelized space to the discrete voxelized space. The resulting voxelized representation of output molecule 162 can be further transformed into reconstructed molecule 650. It should be understood that reconstructed molecule 650 can correspond to either a one-dimensional representation (e.g., a Simplified Molecular Input Line Notation (SMILES) string) or a two-dimensional representation (e.g., a molecular graph) of the output molecule.

[0191] In some exemplary embodiments, the generation performance of the molecular design calculation model 115 may be evaluated based on various metrics, some of which are listed in Table 2 below.

[0192] [Table 2] TIFF2026518659000242.tif69170

[0193] In some exemplary embodiments, the generation performance of the molecular design calculation model 115 is, for example, This may depend on one or more factors, including TIFF2026518659000243.tif5170 and the radius of atomic density within the voxelized molecular representation. Figure 7 shows the voxelized representation of a molecule manipulated by molecular design calculation model 115. When TIFF2026518659000244.tif5170 (e.g., Gaussian noise such as isotropic Gaussian noise) is added, the stability and uniqueness of the molecule generated by molecular design calculation model 115 (Figure 7(a)), the total variation of atoms and the total variation of bonds (Figure 7(b)), and For TIFF2026518659000245.tif7170 (Figure 7(c)) The graph illustrates the effects of TIFF2026518659000246.tif5170. As mentioned above, unlike the diffusion model, according to the various exemplary embodiments described herein, TIFF2026518659000247.tif5170 may be fixed during training and sampling. Furthermore, It should be understood that TIFF2026518659000248.tif5170 is a hyperparameter that imposes a trade-off between the quality of sampling (e.g., gradient-based Markov chain Monte Carlo (MCMC) sampling) and the quality of denoising (e.g., empirical Bayesian framework). In some cases, the molecular design engine 110 is added to the voxelized representation of the molecule, which the denoising engine 117 can still learn to denoise. To accommodate the maximum size of TIFF2026518659000249.tif5170 TIFF2026518659000250.tif5170 can be determined. For example, in some cases, the molecular design calculation model 115 and the noise reduction engine 117 can determine The model may be trained on the QM9 molecular dataset containing TIFF2026518659000251.tif5170, while other hyperparameters are kept constant. The graphs in Figures 7(a), 7(b), and 7(c) show that some metrics are higher. It is improved in TIFF2026518659000252.tif5170, As TIFF2026518659000253.tif5170 increases, the molecular stability and TIFF2026518659000254.tif7170 indicates a decrease. For the QM9 molecular dataset, the best overall performance across all metrics is 0.9. This is achieved in TIFF2026518659000255.tif5170.

[0194] In some exemplary embodiments, it is performed as part of a gradient-based Markov chain Monte Carlo (MCMC) method. TIFF2026518659000256.tif5170 This may affect the novelty of molecules generated by molecular design computation model 115. This phenomenon is shown in Figure 8, which illustrates how molecular design computation model 115 (trained on the Molecular Geometry Ensemble (GEOM) drug dataset) performs. Figure 8 depicts the molecules produced by updating noise molecules (for de novo generation) and known molecules (for seed generation) across TIFF2026518659000257.tif5170. For example, Figure 8 shows the molecular design calculation model 115 Voxelized representation of the first molecule 810, generated by denoising noise molecules (for de novo generation) across sampling iterations of TIFF2026518659000258.tif4170, molecular design calculation model 115 Voxelized representation of the first molecule 820, generated by denoising noise molecules (for de novo generation) across sampling iterations of TIFF2026518659000259.tif4170, molecular design calculation model 115 This shows the voxelized representation of a third molecule, 830, generated by denoising noise molecules (for de novo generation) across sampling iterations of TIFF2026518659000260.tif4170.

[0195] In addition to the novelty of the molecules generated by molecular design calculation model 115, Adjusting TIFF2026518659000261.tif5170 can also affect other aspects of the generation performance of the molecular design calculation model 115. Table 3 below shows compares the generation performance of the molecular design calculation model 115 in TIFF2026518659000262.tif5170 with the generation performance of the conventional generation model EDM that performs 1,000 diffusion steps for generation. The results in Table 3 show that the molecular design calculation model 115 functions better in some measurement criteria as TIFF2026518659000263.tif5170 increases. As predicted, the average time (in seconds) consumed to generate each molecule increases linearly as TIFF2026518659000264.tif5170 increases. However, the molecular design calculation model 115 remains faster than EDM even with 500 sampling iterations. In particular, in just 50 sampling iterations, the molecular design calculation model 115 is already superior to EDM in most measurement criteria and is on average one order of magnitude faster.

[0196]

Table 3

[0197] In some exemplary embodiments, the generation performance of the molecular design calculation model 115 can also be affected by the size of the atomic radius within the voxelized representation in which the molecular design calculation model 115 operates. It should be understood that the size of the atomic radius can change while the resolution of the voxel grid remains fixed (e.g., at 0.25 Å). The generation performance of the molecular design calculation model 115 can peak at a specific atomic radius even with different hyperparameters. For example, when the molecular design calculation model 115 is applied to operate on voxelized representations with atomic radii of 0.25, 0.5, 0.75, and 1.0, the fixed radius of 0.5 was consistently superior to other values even as the hyperparameters of the molecular design calculation model 115 changed.

[0198] In some exemplary embodiments, the generation performance of the molecular design calculation model 115 may be compared to existing generative models that operate on conventional three-dimensional molecular representations, such as GSchNet, a point cloud autoregressive model, and EDM, a point cloud diffusion-based model. Each model is applied to generate 10,000 samples, which are then analyzed for atomic stability, molecular stability, validity, uniqueness, total atomic variability (TV), total bond variability (TV), The evaluation was based on TIFF2026518659000266.tif7170. Table 4 below shows the results for samples generated by Molecular Design Computation Model 115 (MDCM) trained on the QM9 molecular dataset, along with the mean and standard deviation over three runs. Figure 9A shows some examples of voxelized representations of molecules generated by Molecular Design Computation Model 115 trained on the QM9 molecular dataset and their corresponding molecular graphs. The cumulative distribution function (CDF) of strain energy for molecules generated by Molecular Design Computation Model 115 trained on the QM9 molecular dataset, compared to molecules in the QM9 molecular dataset and those generated by the conventional generative model EDM, is shown in graph 1000, depicted in Figure 10A. Figure 10B shows graph 1050, which shows the empirical distribution of the number of atoms per molecule in the QM9 molecular dataset, compared to the empirical distribution of the number of atoms in molecules generated by Molecular Design Computation Model 115 trained on the QM9 molecular dataset.

[0199] [Table 4]

[0200] In some cases, the molecular design computation model 115 is also trained on the Molecular Geometry Ensemble (GEOM) drug dataset before being applied to generate 10,000 samples. A comparison of these samples with the 10,000 samples generated by the conventional generative model EDM is shown in Table 5 below, along with the mean and standard deviation over three separate runs. Figure 9B shows some examples of voxelized representations of molecules generated by the molecular design computation model 115 (MDCM) trained on the Molecular Geometry Ensemble (GEOM) drug dataset and the corresponding molecular graphs. The cumulative distribution function (CDF) of strain energy of molecules generated by the molecular design computation model 115 (MDCM) trained on the GEOM drug dataset, compared to molecules in the Molecular Geometry Ensemble (GEOM) drug dataset and those generated by the conventional generative model EDM, is shown in graph 1100, depicted in Figure 11A. Figure 11B shows Graph 1150, which illustrates the empirical distribution of the number of atoms per molecule in the GEOM drug dataset, compared to the empirical distribution of the number of atoms in molecules generated by a molecular design computation model 115 (MDCM) trained on the GEOM drug dataset.

[0201] [Table 5]

[0202] When Molecular Design Computation Model 115 (MDCM) was trained on the QM9 dataset, it demonstrated comparable generative performance to the conventional generative model EDM. However, when Molecular Design Computation Model 115 (MDCM) was trained on the Molecular Geometry Ensemble (GEOM) drug dataset, a more challenging and realistic drug-like dataset than the QM9 dataset, it demonstrated significantly superior performance to EDM in eight of the nine metrics. For example, molecules generated by Molecular Design Computation Model 115 (MDCM) trained on the GEOM drug dataset showed significantly lower median strain energy than those generated by EDM. Results from Tables 3 and 4 also suggest that augmenting the training dataset with rotation and translation can improve the generative performance of Molecular Design Computation Model 115 (e.g., MDCM). no rot (vs. MDCM). Overall, molecular design computation model 115 is a more expressive model that scales better with the data. In particular, molecular design computation model 115 can better capture many of the modes present in large data distributions such as the Molecular Geometry Ensemble (GEOM) drug dataset.

[0203] Figure 12A depicts a schematic diagram illustrating a comparison of seed generation for molecular geometry ensemble (GEOM) drugs in discrete voxelization space and latent voxelization space over multiple sampling iterations, in several exemplary embodiments. Panel 1210 shows molecular graphs of molecules generated in steps (or sampling iterations) 10, 20, 50, 100, and 200 by molecular design computation model 115, which operates in latent voxelization space and updates the embedded representation of the voxelized representation of seed molecules from the molecular geometry ensemble (GEOM) drug dataset. The corresponding voxelized representations of these molecules are shown in panel 1220. Panel 1215 shows molecular graphs of molecules generated in steps (or sampling iterations) 5, 10, 50, 100, and 200 by molecular design computation model 115, which operates in discrete voxelization space and updates the voxelized representation of seed molecules from the molecular geometry ensemble (GEOM) drug dataset. The corresponding voxelized representations of these molecules are shown in panel 1225. As shown in Figures 12A and 12B, the molecular design computation model 115 can generate stable, reasonable, and unique molecules that closely resemble seed molecules from the Molecular Geometry Ensemble (GEOM) drug dataset, regardless of whether it operates in a latent voxelized space or a discrete voxelized space.

[0204] Table 6 below further illustrates the seed generation results (averaged over 5 iterations) for the Molecular Geometry Ensemble (GEOM) drug dataset.

[0205] [Table 6]

[0206] Figure 12B depicts a schematic diagram illustrating a comparison of seed generation for PubChem drugs in different sampling iterations in discrete voxelization space and latent voxelization space, according to several exemplary embodiments. Panel 1450 shows molecular graphs of molecules generated in steps (or sampling iterations) 10, 20, 50, 100, and 200 by molecular design computation model 115, which operates in latent voxelization space and updates the embedded representation of the voxelized representation of seed molecules from the PubChem dataset. The corresponding voxelized representations of these molecules are shown in panel 1260. Panel 1255 shows molecular graphs of molecules generated in steps (or sampling iterations) 5, 10, 50, 100, and 200 by molecular design computation model 115, which operates in discrete voxelization space and updates the voxelized representation of seed molecules from the PubChem dataset. The corresponding voxelized representations of these molecules are shown in panel 1265. As shown in Figure 12B, the molecular design computation model 115 can also generate stable, reasonable, and unique molecules that closely resemble seed molecules from the PubChem dataset, regardless of whether it operates in a latent voxelization space or a discrete voxelization space.

[0207] Table 7 below further illustrates the seed generation results (averaged over 5 iterations) for the PubChem dataset.

[0208] [Table 7]

[0209] Figure 12C depicts molecular graphs of additional examples of molecules generated in steps (or sampling iterations) 10, 20, 50, 100, and 200 by a molecular design computation model 115 that operates in latent voxelization space and updates the embedding representations of the voxelized representations of two actual drug seed molecules. Figure 12D shows molecular graphs of several exemplary molecules generated in a random selection (or sampling iteration) of steps by a molecular design computation model that operates in latent voxelization space and updates the embedding representations of random molecules (e.g., molecules with randomly selected atomic types and / or positions).

[0210] Table 8 below illustrates the seed generation results (averaged over 5 iterations) for five actual drugs.

[0211] [Table 8]

[0212] Figure 13 shows Graph 1300, which compares over time the number of stable, valid, unique molecules generated by molecular design computation model 115 operating in latent voxelization space, molecular design computation model 115 operating in discrete voxelization space, and state-of-the-art generative model. As shown in Figure 13, molecular design computation model 115 can generate far more stable, valid, unique molecules than state-of-the-art generative model, regardless of whether it is operating in latent voxelization space or discrete voxelization space. Furthermore, molecular design computation model 115 can generate more stable, valid, unique molecules when operating in latent voxelization space than when operating in discrete voxelization space.

[0213] Table 9 below shows molecular design calculation model 115 (MDCM) which performs de novo generation in latent voxelization space. latent ), Molecular design calculation model 115 (MDCM) that performs de novo generation in discrete voxelized space. discreteThis paper compares the generative performance of the state-of-the-art generative model EDM against molecular geometric ensemble (GEOM) drugs (averaged over the generation of 10,000 molecules in three iterations).

[0214] [Table 9]

[0215] Table 10 below shows the molecular design calculation model 115 (MDCM) that performs de novo generation in latent voxelization space. latent ), Molecular design calculation model 115 (MDCM) that performs de novo generation in discrete voxelized space. discrete ), as well as a comparison of the production performance of the state-of-the-art production models GSchNet and EDM on QM9 drugs (averaged over the production of 10,000 molecules repeated three times).

[0216] [Table 10]

[0217] Figure 14 depicts a block diagram showing an example of a computing system 1400 according to several exemplary embodiments. Referring to Figures 1 to 14, the computing system 1400 may be used to implement a molecular design engine 110, a training engine 120, a client device 130, and / or any component thereof.

[0218] As shown in Figure 14, the computing system 1400 may include a processor 1410, memory 1420, storage device 1430, and input / output device 1440. The processor 1410, memory 1420, storage device 1430, and input / output device 1440 may be interconnected via a system bus 1450. The processor 1410 is capable of processing instructions for execution within the computing system 1400. Such executed instructions may implement one or more components, such as a molecular design engine 110, an analysis engine 120, or a client device 130. In some exemplary embodiments, the processor 1410 may be a single-threaded processor. Alternatively, the processor 1410 may be a multi-threaded processor. The processor 1410 is capable of processing instructions stored in memory 1420 and / or storage device 1430 to display graphical information for a user interface provided via the input / output device 1440.

[0219] Memory 1420 is a computer-readable medium, such as volatile or non-volatile, that stores information within the computing system 1400. Memory 1420 can store, for example, data structures representing a configuration object database. Storage device 1430 is capable of providing persistent storage for the computing system 1400. Storage device 1430 may be a floppy disk device, a hard disk device, an optical disk device, or a tape device, or alternatively, another suitable means of persistent storage. Input / output device 1440 provides input / output operation for the computing system 1400. In some exemplary embodiments, input / output device 1440 includes a keyboard and / or a pointing device. In various embodiments, input / output device 1440 includes a display unit for displaying a graphical user interface.

[0220] According to some exemplary embodiments, the input / output device 1440 may provide input / output operation for network devices. For example, the input / output device 1440 may include an Ethernet port or other networking port to communicate with one or more wired and / or wireless networks (e.g., a local area network (LAN), a wide area network (WAN), the Internet).

[0221] In some exemplary embodiments, the computing system 1400 may be used to run various interactive computer software applications that can be used for organizing, analyzing, and / or storing various forms of data. Alternatively, the computing system 1400 may be used to run any type of software application. These applications may be used to perform various functions, such as planning functions (e.g., generating, managing, and editing spreadsheet documents, word processing documents, and / or any other objects), computing functions, communication functions, etc. An application may include various add-in functions or may be a standalone computing product and / or function. When activated within an application, functionality may be used to generate a user interface provided via the input / output device 1440. The user interface may be generated by the computing system 1400 and presented to the user (e.g., on a computer screen monitor).

[0222] One or more aspects or features of the subject matter described herein may be realized in digital electronic circuits, integrated circuits, specially designed ASICs, field-programmable gate array (FPGA) computer hardware, firmware, software, and / or combinations thereof. These various aspects or features may include implementations in one or more computer programs executable and / or interpretable on a programmable system which includes at least one programmable processor, which may be special or general-purpose, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to transmit data and instructions to the storage system, at least one input device, and at least one output device. The programmable system or computing system may include clients and servers. Clients and servers are generally remote from each other and generally interact over a communication network. The relationship between clients and servers is brought about by computer programs running on each computer and having a client-server relationship with each other.

[0223] These computer programs, also called programs, software, software applications, applications, components, or code, contain machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the term “machine-readable medium” means any computer program product, apparatus, and / or device used to provide machine instructions and / or data to a programmable processor, such as magnetic disks, optical disks, memory, and programmable logic devices (PLDs), and includes machine-readable medium that receives machine instructions as machine-readable signals. The term “machine-readable signals” means any signals used to provide machine instructions and / or data to a programmable processor. Machine-readable medium can store such machine instructions non-temporarily, such as non-temporarily solid-state memory, magnetic hard drives, or any equivalent storage medium. Machine-readable medium can, alternatively or additionally, store such machine instructions temporarily, such as processor caches or other random-access memories associated with one or more physical processor cores.

[0224] To provide user interaction, one or more aspects or features of the subject matter described herein may be implemented on a computer having, for example, a display device such as a cathode ray tube (CRT), liquid crystal display (LCD), or light-emitting diode (LED) monitor for displaying information to the user, and a keyboard and a pointing device such as a mouse or trackball, to which the user can provide input to the computer. User interaction may also be provided using other types of devices. For example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and input from the user may be received in any form, including acoustic, speech, or tactile input. Other possible input devices include touchscreens, or single-point or multi-point resistive or capacitive trackpads, speech recognition hardware and software, optical scanners, optical pointers, digital image capture devices, and other touch-sensitive devices such as associated interpretation software.

[0225] In the above description and claims, phrases such as “at least one of ~” or “one or more of ~” may appear, followed by a conjunctive list of elements or features. The term “and / or” may also be used in the enumeration of two or more elements or features. Unless implicitly or explicitly contradicted by the context in which it is used, such phrases are intended to mean either any of the enumerated elements or features individually, or any of the enumerated elements or features in combination with any of the other enumerated elements or features. For example, the phrases “at least one of A and B,” “one or more of A and B,” and “A and / or B” are intended to mean “A only,” “B only,” or “A and B together,” respectively. The same interpretation is intended for lists containing three or more items. For example, the phrases “at least one of A, B, and C;”, “one or more of A, B, and C;”, and “A, B, and / or C” are intended to mean “A alone, B alone, C alone, A and B, A and C, B and C, or A, B and C,” respectively. The use of the term “based on” in the foregoing and in the claims is intended to mean “at least partially based,” so as to allow for features or elements that are not enumerated.

[0226] The subject matter described herein may be implemented in systems, apparatus, methods, and / or articles, depending on the desired configuration. The implementations described above do not necessarily represent all implementations of the subject matter described herein. Rather, those embodiments are merely examples that correspond to aspects associated with the described subject matter. While some variations have been described in detail above, other modifications or additions are also possible. In particular, further features and / or variations may be provided in addition to those described herein. For example, the embodiments described above may cover various combinations and partial combinations of the disclosed features, and / or combinations and partial combinations of some of the further features described above. Furthermore, the logical flows shown in the accompanying drawings and / or described herein do not necessarily require a specific order or sequence shown to achieve the desired result. Other embodiments may fall within the scope of the following claims.

Claims

1. A computer implementation method, Encoding the voxelized representation of the input molecule to generate an embedded representation of the input molecule having fewer features than the voxelized representation of the input molecule, Applying a molecular design calculation model to update the embedding representation of the input molecule, The molecular design calculation model identifies a data distribution of molecules that exhibit one or more desirable properties. The corrupted embedded representation of the voxelized representation of a sample molecule exhibiting one or more of the aforementioned desired properties is taken as input. It is trained to approximate by reconstructing the embedding representation of the voxelized representation of the sample molecule from the damaged embedding representation. The molecular design calculation model updates and applies the embedding representation of the input molecule such that the resulting updated embedding representation increases the likelihood that it falls within the data distribution. A computer implementation method comprising generating a voxelized representation of an output molecule by decoding at least the updated embedding representation obtained as a result.

2. The method according to claim 1, wherein the data distribution is a noisy data distribution occupied by noisy embedding representations of voxelized representations of the molecules exhibiting one or more desirable characteristics, and the voxelized representation is further generated by denoising the noisy voxelized representation of the output molecules generated by decoding the resulting updated embedding representation.

3. The method according to claim 1 or 2, further comprising applying a vector quantization variational autoencoder (VQ-VAE) to encode the voxelized representation of the input molecule and to decode the resulting updated embedding representation.

4. The method according to claim 3, wherein the molecular embedding representation is a discrete latent embedding vector generated by quantizing a corresponding continuous latent embedding representation, and the quantization includes matching the corresponding continuous latent embedding representation with a vector in the embedding representation's codebook by nearest neighbor search.

5. The method according to any one of claims 1 to 4, wherein the voxelized representation of the input molecule is encoded by compressing a plurality of atomic density values ​​including the voxelized representation of the input molecule such that the voxelized representation of the input molecule includes at least fewer features than the embedding representation of the input molecule.

6. The method according to any one of claims 1 to 5, wherein the voxelized representation of a molecule comprises a plurality of voxels organized into a three-dimensional voxel grid, and each atom in the molecule is represented as a continuous density across one or more voxels in the three-dimensional voxel grid.

7. The method according to claim 6, wherein the continuous density of each atom in the molecule is centered at the center of each atom, and a first voxel located away from any atom in the molecule is associated with a lower atomic density value than a second voxel located closer to the center of the atom in the molecule.

8. The method according to any one of claims 6 to 7, wherein each voxel in the three-dimensional voxel grid is associated with a value indicating the atomic density at a corresponding position.

9. The method according to any one of claims 1 to 8, wherein the voxelized representation of the molecule comprises one or more channels, each channel corresponding to a type of atom present in the molecule.

10. The method according to any one of claims 1 to 9, wherein the voxelized representation of the molecule represents both the type and position of one or more atoms present in the molecule.

11. The method according to any one of claims 1 to 10, wherein the embedding representation of the input molecule is updated based on a function parameterized by at least a plurality of parameters of the molecular design calculation model, and the function outputs a value indicating the likelihood that the resulting updated embedding representation falls within the data distribution.

12. The method according to claim 11, wherein the function is a score function, and the value output by the function is a score indicating a local change in the density of the noisy data distribution at the location of each updated noisy embedding representation generated by the update of the noisy embedding representation of the input molecule.

13. The molecular design calculation model uses, at least, the embedding representation of the input molecule. The molecular design calculation model is applied to update the embedding representation of the input molecule, thereby generating a first updated embedding representation. The molecular design calculation model is applied to update the embedding representation of the input molecule, thereby generating a second updated embedding representation. Applying a function parameterized by a plurality of parameters of the molecular design calculation model to determine (i) a first value representing a first local change in the density of the data distribution at a first location occupied by the first updated embedding representation, and (ii) a second value representing a second local change in the density of the data distribution at a second location occupied by the second updated embedding representation, The method according to any one of claims 1 to 12, wherein the molecular design calculation model is applied to further update the first updated embedding representation in place of the second updated embedding representation based on at least the first and second values.

14. The method according to claim 13, wherein the molecular design calculation model is applied to further update the first updated embedding representation until one or more criteria are met, the one or more criteria being at least one of (i) the updating of the embedding representation of the input molecule has been performed a threshold number of times, (ii) the first value of the first updated embedding representation satisfies one or more thresholds, and (iii) a threshold amount of output molecule has been generated.

15. The method according to claim 13 or 14, wherein the molecular design calculation model is applied to further modify the first updated embedding representation in place of the second updated embedding representation, based on the fact that at least the first value and the second value indicate that the first updated embedding representation is more likely to be included in the data distribution than the second updated embedding representation.

16. The method according to any one of claims 13 to 15, wherein the molecular design calculation model is applied to further modify the first updated embedding representation in place of the second updated embedding representation, based on the fact that at least the first and second values ​​indicate that the first updated embedding representation is sampled from a higher density region of the data distribution than the second updated embedding representation.

17. The method according to any one of claims 1 to 16, further comprising converting the voxelized representation of the output molecule into a one-dimensional representation and / or a two-dimensional representation of the output molecule.

18. The voxelized representation of the output molecule is at least, The position of one or more atoms in the output molecule is determined by detecting one or more peaks in a plurality of atomic density values ​​that include at least the voxelized representation of the output molecule, The method according to claim 17, which is transformed by determining one or more bonds based on the positions of at least one or more atoms.

19. It is a system, At least one data processor, A memory that stores instructions which, when executed by the at least one data processor, result in an operation including the method according to any one of claims 1 to 18, A system that includes these features.

20. A non-temporary computer-readable medium that stores instructions that, when executed by at least one data processor, result in an operation including the method according to any one of claims 1 to 18.

21. A computer implementation method, To generate a training dataset containing multiple training samples, wherein each training sample in the training dataset includes a corrupted embedding representation generated by adding at least noise to a noisy voxelized representation of a sample molecule exhibiting one or more desirable characteristics. Training a molecular design computation model to approximate the data distribution of one or more molecules exhibiting the desired properties, based on at least the training dataset, wherein the training includes applying the molecular design computation model to reconstruct an undamaged embedding representation of the noisy voxelized representation of the sample molecule from the damaged embedding representation of the noisy voxelized representation of the sample molecule; A computer implementation method comprising, optionally, applying the molecular design calculation model to generate an output molecule by at least denoising the embedding representation of the voxelized representation of the input molecule and decoding the resulting updated embedding representation to generate a voxelized representation of the output molecule.

22. The method according to claim 21, wherein the noisy voxelized representation of the sample molecule comprises a plurality of voxels organized into a three-dimensional voxel grid, and each atom in the sample molecule is represented as a continuous density across one or more voxels in the three-dimensional voxel grid.

23. The method according to claim 22, wherein the continuous density of each atom in the sample molecule is centered at the center of each atom, and a first voxel located away from any atom in the sample molecule is associated with a lower atomic density value than a second voxel located closer to the center of the atom in the sample molecule.

24. The method according to claim 22 or 23, wherein each voxel in the three-dimensional voxel grid is associated with a value indicating the atomic density at a corresponding position.

25. The method according to any one of claims 21 to 24, wherein the noisy voxelized representation of the sample molecule includes one or more channels, each channel corresponding to a type of atom present in the sample molecule.

26. The method according to any one of claims 21 to 25, wherein the noisy voxelized representation of the sample molecule represents both the type and position of one or more atoms present in the sample molecule.

27. The method according to any one of claims 21 to 26, wherein the training of the molecular design calculation model includes adjusting a plurality of parameters of the molecular design calculation model to reduce the difference between the restored embedding representation generated by the molecular design calculation model and the undamaged embedding representation of the noisy voxelized representation of the sample molecule.

28. The method according to claim 27, wherein the plurality of parameters of the molecular design calculation model parameterize a function, and the plurality of parameters are adjusted to output values ​​that indicate local changes in the density of the data distribution of molecules in which the function exhibits one or more desirable properties.

29. The method according to claim 1, wherein the molecular design calculation model denoises the embedding representation of the voxelized representation of the input molecule by updating at least one value present in the embedding representation that represents a plurality of atomic density values ​​present in the voxelized representation of the input molecule.

30. The method according to claim 29, wherein updating the atomic density values ​​of one or more voxels in the at least one channel of the voxelized representation of the input molecule corresponds to updating at least one of the types and / or positions of one or more atoms present in the input molecule.

31. The method according to any one of claims 21 to 30, wherein the molecular design calculation model denoises the embedding representation of the voxelized representation of the input molecule over multiple iterations of gradient-based Markov chain Monte Carlo (MCMC) sampling until one or more criteria are met.

32. The method according to claim 31, wherein one or more of the criteria include at least one of the following: (i) a gradient-based Markov chain Monte Carlo (MCMC) sampling iteration has been performed a threshold number of times; (ii) the resulting updated embedding representation has been sampled from a region having a threshold density; and (iii) a threshold output molecule has been generated.

33. The molecular design calculation model determines the voxelized representation of the output molecule, at least: Applying a first update to the embedding representation of the voxelized representation of the input molecule to generate a first updated embedding representation, Applying a second update to the embedding representation of the voxelized representation of the input molecule to generate a second updated embedding representation, The method according to any one of claims 21 to 32, wherein, upon determining that the first updated embedding representation is sampled from a higher density region of the data distribution than the second updated embedding representation, the first updated embedding representation is further updated in place of the second updated embedding representation.

34. The method according to any one of claims 21 to 33, further comprising converting the voxelized representation of the output molecule into a one-dimensional representation and / or a two-dimensional representation of the output molecule.

35. The training of the molecular design calculation model is To generate a first restored embedding representation of the noisy voxelized representation of the sample molecule, the molecular design calculation model having a first adjustment is applied, Determine a first mean squared error (MSE) that quantifies the first difference between the first restored embedding representation and the undamaged embedding representation of the noisy voxelized representation of the sample molecule, To generate a second restored embedding representation of the noisy voxelized representation of the sample molecule, the molecular design calculation model having a second adjustment is applied, Determine a second mean squared error (MSE) that quantifies the second difference between the second restored embedding representation and the undamaged embedding representation of the noisy voxelized representation of the sample molecule, The method according to any one of claims 21 to 34, further adjusting the molecular design calculation model having the first adjustment instead of the second adjustment, upon determining that the first mean squared error (MSE) is smaller than the second mean squared error (MSE).

36. The method according to claim 35, wherein the molecular design calculation model is further adjusted until one or more criteria are met, the one or more criteria being at least one of (i) the adjustment of the molecular design calculation model has been performed a threshold number of times, and (ii) a restored embedding representation exhibiting a threshold mean squared error (MSE) value has been generated.

37. The method according to any one of claims 21 to 36, further comprising training an autoencoder comprising an encoder and a decoder, wherein the training of the autoencoder includes training the encoder to encode the noisy voxelized representation of the sample molecule such that the decoder can reconstruct the voxelized representation of the sample molecule from an embedding representation obtained as a result of the noisy voxelized representation of the sample molecule.

38. The method according to claim 37, wherein the autoencoder is a vector quantization variational autoencoder (VQ-VAE), and in the VQ-VAE, the encoder generates a continuous latent embedding representation of the sample molecule, and the continuous latent embedding representation is then quantized by matching it with a vector in the embedding representation codebook by nearest neighbor search to obtain a discrete latent embedding representation for decoding by the decoder.

39. It is a system, At least one data processor, A memory that stores instructions which, when executed by the at least one data processor, result in an operation including the method according to any one of claims 21 to 38, A system that includes these features.

40. A non-temporary computer-readable medium that stores instructions that, when executed by at least one data processor, result in an operation including the method according to any one of claims 21 to 38.

Images