Molecular Design Using Reinforcement Learning

Abstract

Methods, apparatuses, and systems are provided for designing compounds that exhibit one or more desired properties using machine learning (ML) techniques. This can be achieved by generating a second compound using an ML technique that modifies a first compound based on the desired property(s) and a set of rules for modifying the compound; scoring the second compound based on the desired property(s); determining whether to repeat the generating step based on the score; and updating the ML technique based on the score before repeating the generating step.

Description

Technical Field

[0001] The present application relates to devices, systems, and methods for using machine learning techniques to design chemical compounds that exhibit one or more desired properties. Background Art

[0002] Informatics is the application of computers, information technology, and resources to interpret data in one or more academic and / or scientific fields. Cheminformatics and bioinformatics include the application of computers, information technology, and resources to interpret chemical and / or biological data. This can include solving and / or simulating processes and / or problems in chemistry and / or biology. For example, these computing, information technology, and resources can convert data into information, and subsequently convert information into knowledge, which can be used to rapidly generate compounds and / or make improved decisions, in the fields of drug identification, development, and optimization, by way of example only and not limitation.

[0003] Machine learning techniques are computational methods that can be used to design complex analytical models and algorithms suitable for solving complex problems, such as generating or predicting compounds with desired characteristics and / or properties. There are a variety of machine learning techniques that can be used or selected for generating compounds, but none can be used to generate or optimize compounds based on a set of desired characteristics or properties. It is expected that machine learning techniques will enable researchers, data scientists, engineers, and analysts to achieve rapid improvements in the fields of drug identification, development, and optimization.

[0004] The embodiments described below are not limited to implementations that solve part or all of the deficiencies of the above-mentioned known methods. Summary of the Invention

[0005] This section is intended to introduce in simplified form some of the concepts that will be further described in the detailed description section below. This section is not intended to identify key features or essential features of the subject matter described, nor is it intended to be used to determine the scope of the subject matter described; variations and alternative features that facilitate the operation of the present invention and / or achieve substantially similar technical effects should be considered to fall within the scope of the invention disclosed herein.

[0006] The present disclosure provides methods, devices, and systems for designing and generating candidate molecules / compounds that exhibit desired properties using machine learning techniques. Machine learning techniques can be used to learn how to perturb an initial compound until a compound with the desired properties is achieved or a stopping criterion is reached. The methods described herein do not require human intervention and can be executed automatically. The sequence of rules, operations, and / or perturbations that resulted in the final compound can also be output, providing researchers with additional insights and / or evidence on how the compound was formed and whether the compound possessed the desired properties and / or characteristics.

[0007] In a first aspect, the present disclosure provides a computer-implemented method for designing compounds that exhibit one or more desired properties using machine learning techniques, the method comprising: generating a second compound using the machine learning technique to modify a first compound based on the desired properties and a set of rules for modifying the compound; scoring the second compound based on the desired properties; determining whether to repeat the generating step based on the score; and updating the machine learning technique based on the score before repeating the generating step.

[0008] Preferably, determining whether to repeat the generating step is based on a score indicating that the second compound is closer to the compound exhibiting the desired property.

[0009] Preferably, determining whether to repeat the generating step is based on a score indicating that the second compound exhibits the desired property.

[0010] Preferably, determining whether to repeat the generating step further comprises determining whether a predetermined number of iterations of repeating the generating step have been achieved.

[0011] Preferably, determining whether to repeat the generating step further comprises determining whether any further improvements to the second compound are possible based on the second compound exhibiting at least one or more desired properties.

[0012] Preferably, generating the second compound further comprises generating a second set of compounds; and scoring the second set of compounds based on the desired properties.

[0013] Preferably, the method further comprises: ranking the second compound set based on the scores, wherein generating the second compound further comprises generating other second compounds based on the second compound set with the highest hierarchy.

[0014] Preferably, the rule set also includes data representing one or more operations associated with modifying the compound.

[0015] Preferably, the one or more operations include one or more operations from the following group: operations corresponding to adding a compound fragment or one or more atoms to a compound; operations corresponding to removing a compound fragment or one or more atoms of a compound; operations corresponding to breaking or removing bonds between atoms of a compound; operations corresponding to adding or reforming bonds between atoms of a compound; any other operations associated with modifying a compound to form another compound; and any other operations associated with modifying a compound to form a different compound.

[0016] Preferably, the rule set and / or the one or more operations are selected to conform to desired structural, physical and / or chemical constraints that ensure that any modification of the compound and / or subsequently modified compound is feasible.

[0017] Preferably, the set of rules and / or the one or more operations are based on a set of related chemical groups, which includes one or more of the following: one or more atoms; one or more molecules; one or more other compounds; one or more compound fragments; one or more bonds; one or more functional groups; and one or more chemically related aspects of the compound, etc.

[0018] Preferably, generating the second compound further comprises generating a tree data structure comprising a plurality of nodes and a plurality of edges, wherein each edge connects a parent node to a child node, wherein the parent node represents a compound, and each edge from the parent node to the child node represents an operation of a plurality of operations performed on the compound of the parent node that results in the formation of the compound of the child node, wherein the root node of the tree is the first compound, and subsequent nodes correspond to the second set of compounds.

[0019] Preferably, the method further comprises expanding the tree data structure based on scoring one or more nodes corresponding to the second set of compounds.

[0020] Preferably, the method further comprises performing a tree search on the tree data structure to generate a second set of compounds based on a set of one or more operations from the plurality of operations.

[0021] Preferably, generating one or more second compounds also includes: mapping the first compound and the set of operations to an N-dimensional operation space through machine learning technology; selecting a subset of the nearest neighbor operations of the first compound in the N-dimensional operation space when mapped in the N-dimensional operation space through machine learning technology; and applying the subset of operations in the N-dimensional space to the first compound to generate a set of one or more second compounds.

[0022] Preferably, generating the second compound set further comprises selecting nodes associated with the selected operation set for inclusion in the tree data structure.

[0023] Preferably, the desired properties include one or more from the following group: the compound docks with another compound to form a stable complex; a specific property is associated with a ligand that docks to a target protein, wherein the compound is the ligand; the compound docks or binds to one or more target proteins; the compound has a specific solubility or solubility range; and any other property associated with the compound that can be simulated using computer simulations based on the physical movement of atoms and molecules.

[0024] Preferably, the score includes a certainty score, wherein one or more of the second compounds has a higher certainty score when one or more of the second compounds substantially exhibit all of the one or more desired properties; one or more of the second compounds has a lower certainty score when one or more of the second compounds substantially do not exhibit some of the one or more desired properties; and one or more of the second compounds has an uncertainty score between a higher certainty score and a lower certainty score when the second compound substantially exhibits some of the one or more desired properties.

[0025] Preferably, the certainty score is a percentage certainty score, wherein a higher certainty score is 100%, a lower certainty score is 0%, and the uncertainty score is between the higher and lower certainty scores.

[0026] Preferably, generating one or more second compounds further comprises using reinforcement learning techniques to select one or more of a plurality of rules for modifying the first compound into the second compound.

[0027] Preferably, at least part of the scoring is performed using one or more machine learning techniques.

[0028] Preferably, the machine learning technology includes at least one machine learning technology or a combination of machine learning technologies from the following group: a recursive neural network, which is configured to predict a second compound that exhibits a desired set of properties starting from a first compound; a convolutional neural network, which is configured to predict a second compound that exhibits a desired set of properties starting from a first compound; a reinforcement learning algorithm, which is configured to predict a second compound that exhibits a desired set of properties starting from a first compound; and any neural network structure configured to predict a second compound that exhibits a desired set of properties starting from a first compound.

[0029] Preferably, scoring the second compound based on the desired properties further comprises: analyzing the second compound for each of the desired properties; and calculating an aggregate score for the second compound based on the analysis.

[0030] Preferably, analyzing the second compound further comprises performing computer simulations related to one or more desired properties of the second compound.

[0031] Preferably, analyzing the second compound further comprises using a knowledge-based expert system to determine whether the second compound exhibits one or more desired properties.

[0032] Preferably, when a machine learning technique is used to generate a second compound, one or more first compounds are input to the machine learning technique.

[0033] Preferably, generating the second compound using machine learning techniques further comprises generating a second set of compounds based on the desired properties and the rule set using machine learning techniques.

[0034] In a second aspect, the present disclosure provides an apparatus comprising a processor, a memory unit and a communication interface, wherein the processor is connected to the memory unit and the communication interface, wherein the processor and the memory are configured to implement a computer-implemented method according to the first aspect, a modification thereof and / or as described herein.

[0035] In a third aspect, the present disclosure provides a computer readable medium comprising data or instruction code which, when executed on a processor, causes the processor to implement a computer-implemented method according to the first aspect, a modification thereof and / or as described herein.

[0036] In a fourth aspect, the present disclosure provides a machine learning model comprising data representing updating a machine learning technique according to the first aspect, a modification thereof and / or a computer-implemented method as described herein.

[0037] In a fifth aspect, the present disclosure provides a machine learning model obtained from a computer-implemented method according to the first aspect, a modification thereof and / or as described herein.

[0038] In a sixth aspect, the present disclosure provides a tangible computer-readable medium comprising data or instruction code for using machine learning techniques to design compounds that exhibit one or more desired properties, which, when executed on one or more processors, causes one of the one or more processors to perform at least one step of the following method: generating a second compound using a machine learning technique that modifies a first compound based on the desired properties and a set of rules for modifying the compound; scoring the second compound based on the desired properties; determining whether to repeat the generating step based on the score; and updating the machine learning technique based on the score before repeating the generating step.

[0039] Preferably, the computer readable medium further comprises data or instruction codes which, when executed on a processor, cause the processor to implement one or more steps of the computer-implemented method according to the first aspect, modifications thereof and / or as described herein.

[0040] In a seventh aspect, the present disclosure provides a system for using machine learning technology to design compounds that exhibit one or more desired properties, the system comprising: a compound generation module configured to generate a second compound using a machine learning technology that modifies a first compound based on the desired properties and a set of rules for modifying the compound; a compound scoring module configured to score the second compound based on the desired properties; a determination module configured to determine whether to repeat the generation step based on the score; and an update machine learning module configured to update the machine learning technology based on the score before repeating the generation step.

[0041] Preferably, the compound generation module, the compound scoring module, the decision module and the update machine learning module are further configured to implement the computer-implemented method according to the first aspect, modifications thereof and / or as described herein.

[0042] The methods described herein may be performed by software in a machine-readable form on a tangible storage medium, for example in the form of a computer program comprising computer program code means adapted to perform all of the steps of any of the methods described herein when the program is run on a computer, where the computer program can be implemented on a computer-readable medium. Examples of tangible (or non-transitory) storage media include disks, thumb drives, memory cards, and the like, and do not include propagating signals. The software may be adapted to be executed on a parallel processor or a serial processor, such that the method steps may be performed in any suitable order or simultaneously.

[0043] This application recognizes that firmware and software can be valuable, individually tradable commodities. This is intended to include software that runs on or controls "dumb" or standard hardware to perform a desired function. It is also intended to include software that "describes" or defines the configuration of hardware, such as HDL (Hardware Description Language) software, which is used to design silicon chips, or to configure general-purpose programmable chips to perform a desired function.

[0044] As will be apparent to those skilled in the art, the preferred features may be combined as appropriate and with any aspect of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Embodiments of the present invention will be described by way of example with reference to the following drawings, in which:

[0046] Figure 1a is a flow chart illustrating an exemplary process for designing compounds that exhibit one or more desired properties according to the present invention;

[0047] Figure 1b is a flow chart illustrating an exemplary apparatus for designing compounds that exhibit one or more desired properties according to the present invention;

[0048] Figure 2a is a flow chart illustrating an exemplary process for a machine learning technique for generating compounds that exhibit one or more desired properties in accordance with the present invention;

[0049] Figure 2b is a schematic diagram of an exemplary apparatus illustrating a machine learning technique for generating compounds exhibiting one or more desired properties in accordance with the present invention;

[0050] Figure 3a is a schematic diagram illustrating an exemplary tree-based data structure used by machine learning techniques according to the present invention;

[0051] Figure 3b is a flow chart illustrating an exemplary process for a tree-based machine learning technique for generating compounds that exhibit one or more desired properties in accordance with the present invention;

[0052] Figures 3c to 3d According to the present invention, Figure 3b A schematic diagram of the operation of an exemplary tree-based machine learning technique for the process;

[0053] Figure 4a is a flow chart illustrating an exemplary process for generating compounds that exhibit one or more desired properties based on encoding machine learning techniques according to the present invention;

[0054] Figures 4b to 4e is a method according to the present invention for producing one or more candidate compounds that exhibit one or more desired properties. Figure 4a a schematic diagram of the operation of the process;

[0055] Figure 5a is a method according to the present invention for producing one or more candidate compounds that exhibit one or more desired properties. Figures 3a to 4e A flowchart of an exemplary process of a tree-encoding machine learning technique;

[0056] Figure 5b is a method according to the present invention for producing one or more candidate compounds that exhibit one or more desired properties. Figure 5a is a schematic representation of the operation of a rational process;

[0057] Figure 6a is a schematic diagram of a computing system and device according to the present invention; and

[0058] Figure 6b is a schematic diagram of a system according to the present invention.

[0059] The same reference numerals are used throughout the drawings to denote the same features. DETAILED DESCRIPTION

[0060] The following description of embodiments of the present invention is provided by way of example only. These examples represent the best modes currently known to applicants for implementing the present invention, but are not the only means of implementing the present invention. The description provides the functions of the examples, as well as the sequence of steps for constructing and operating the examples. However, the same or equivalent functions and sequences may be achieved through different examples.

[0061] The present inventors have advantageously discovered an iterative and automated method using machine learning techniques for designing and generating candidate molecules / compounds that exhibit desired properties based on starting with an initial compound and perturbing it until a compound with the desired properties can be achieved. The method does not require human intervention and can be performed automatically. The sequence of rules, operations, and / or perturbations performed that lead to the final compound can also be output and provide researchers with additional insights and / or evidence on how the compound was formed and that the compound has the desired properties and / or characteristics.

[0062] For example, reinforcement learning (RL) techniques can be applied that use one or more machine learning techniques, by way of example only and not limitation, neural networks, to design and generate new molecules / compounds that exhibit one or more desired properties. Reinforcement learning techniques use machine learning techniques to iteratively generate a sequence of operations for modifying an initial compound molecule / compound into another molecule / compound that may exhibit the desired properties. The machine learning technique can be configured to apply all known or possible operations (e.g., adding atoms, breaking bonds, removing atoms, etc.) that it can perform on the initial molecule / compound or its fragments and the desired properties to output one or more possible candidate compounds.

[0063] Each candidate compound may be scored based on, by way of example only and not limitation, atomistic computer simulations (e.g., molecular dynamics (RTM)) and / or a knowledge-based expert system, or one or more machine learning techniques trained to score compounds for one or more desired properties, to determine whether the candidate compound is already known and how close it is to exhibiting the desired property. The reinforcement learning technique updates or adjusts the machine learning technique based on the score. The update of the machine learning technique may include, by way of example only and not limitation, updating or adjusting the parameters, coefficients, and / or weights of the machine learning technique. During the update, the reinforcement learning technique may penalize the machine learning technique if the desired property moves further away from the starting molecule / compound, or if the modified molecule / compound is too large / too small, and / or exhibits any other undesirable qualities or differences. The reinforcement learning technique may reward the machine learning technique if the modified molecule exhibits properties that are closer to the desired property. The reinforcement learning technique then iterates the design process again, which may include the machine learning technique, starting again from the starting compound and / or from one of the output candidate compounds, and applying another sequence of operations to obtain another modified molecule / compound. By way of example only and not limitation, the iterative process of the reinforcement learning technique may terminate when one of the following conditions is reached: a maximum number of iterations have occurred, no further significant improvements in the candidate compounds occur (e.g., observed when the scores plateau compared to previous iterations), the scores indicate that one or more candidate compounds exhibit a desired property and / or no further significant improvements in the candidate compounds occur.

[0064] A compound (also referred to as one or more molecules) may comprise or represent a chemical or biological substance composed of one or more molecules (or molecular entities) composed of atoms from one or more chemical elements (or more than one chemical element) held together by chemical bonds. Exemplary compounds as used herein may include, by way of example only and not limitation, molecules held together by covalent bonds, ionic compounds held together by ionic bonds, intermetallic compounds held together by metallic bonds, certain complexes held together by coordinate covalent bonds, pharmaceutical compounds, biological compounds, biomolecules, biochemical compounds, one or more proteins or protein compounds, one or more amino acids, lipids or lipid compounds, carbohydrates or synthetic carbohydrates, nucleic acids, deoxyribonucleic acid (DNA), DNA molecules, ribonucleic acid (RNA), RNA molecules, and / or any other organization or structure of molecules or molecular entities composed of atoms from one or more chemical elements, and combinations thereof.

[0065] Each compound has or exhibits one or more properties, characteristics, or traits, or a combination thereof, that determine the compound's usefulness for a given application. A property of a compound may include or represent data that indicates or is indicative of a particular behavior / characteristic / trait and / or property of the compound when undergoing a reaction. A desired property of a compound may include or represent data that indicates one or more, or a range of, characteristics, traits, and / or properties that the compound is desired to exhibit or be associated with. A compound may exhibit or be associated with one or more properties, which may include, but is not limited to, aspects of one or more characteristics, traits, and / or properties from the following group: an indication that the compound docks with another compound to form a stable complex; an indication associated with docking a ligand to a target protein, where the compound is the ligand; an indication that the compound docks or binds to one or more target proteins; an indication that the compound has a particular solubility or solubility range; an indication that the compound has a particular electrical property; an indication that the compound has toxicity or a range of toxicity; any other indication of a property or characteristic associated with the compound that can be simulated using computer simulations based on the physical movement of atoms and molecules; or any other indication of a property or characteristic associated with the compound that can be experimentally tested or measured. Other examples of one or more compound properties, characteristics, or traits may include, by way of example only and not limitation, aspects of one or more of the following: partition coefficient (e.g., logP), distribution coefficient (e.g., logD), solubility, toxicity, drug-target interaction, drug-drug interaction, off-target drug effect, cell penetration, tissue penetration, metabolism, bioavailability, excretion, absorption, distribution, drug-protein binding, drug-protein interaction, drug-lipid interaction, drug-DNA / RNA interaction, metabolite prediction, tissue distribution, and / or any other suitable compound-related property, characteristic, and / or trait.

[0066] A compound that exhibits a property or is associated with a property can be represented by a property value or property score of the compound, which may include or contain data representing or indicating whether the compound exhibits or is associated with a specific behavior / characteristic / trait of the property when the compound undergoes a reaction when tested for the property. The relationship between the compound and the property can be tested via laboratory experiments, computer simulations (e.g., physical or atomic computer simulations), or via a machine learning model configured to predict whether the compound exhibits the tested property or is associated with the tested property. Therefore, the property value or property score data representing or indicating the compound that exhibits or is associated with the property can be based on measured values, simulation result values ​​or data, and / or machine learning model output values ​​and / or predicted result values / scores and / or data, etc. The property value / score of a compound associated with a property may include or represent data representing or indicating whether the compound exhibits the property or is associated with the property. The property value / score may include, by way of example only and not limitation, data representing any one or more continuous property values / scores (e.g., non-binary values), one or more discrete property values / scores (e.g., binary values ​​or integer values), one or more ranges of continuous property values / scores, one or more ranges of discrete property values / scores, a series of property values / scores, one or more strings of property values, a measurement associated with a reaction, or any other suitable data representation of a property value / score indicating whether a compound exhibits the property or the degree to which it is associated with the property. For example, a compound may undergo a reaction associated with a property and, based on a measurement or simulation, be assigned a property value / score indicating whether the compound exhibits the property or the degree to which it is associated with the property. The property value / score may be based on measurement data or simulation data associated with the reaction and / or the particular property.

[0067] Therefore, it is possible to score a compound for one or more desired properties based on one or more property values / scores of a determinable compound. The overall property score of a compound of a desired property set can be based on a combination of the individual property values / scores of the desired property set, and the compound is tested for the desired property set. Any other method or the like for determining whether a compound exhibits one or more properties can be used (as an example but not limited thereto), laboratory experiments, computer simulations, machine learning model predictions, and / or the like, to test the compound for each individual property in the desired property set. For example, the combination of property values / scores can be a weighted combination of the individual properties of the desired property set. The overall compound property score obtained that can be assigned to the compound gives an indication of whether the compound is associated with the desired property set or has exhibited the desired property set.

[0068] A rule for modifying a compound may include or represent data representing any principle, operation, rule, procedure, operation, or any other command, code, or instruction or data format that can be used to describe modifying a compound from a first compound to a second compound. A rule set may include or represent data representing one or more rules, or multiple rules, for modifying a compound. For example, a rule set for modifying a compound may include or represent data representing one or more operations associated with modifying a compound, such as, but not limited to, one or more rules or operations from the following group: a rule or operation corresponding to adding a chemical element, a compound fragment, or one or more atoms to a compound; a rule or operation corresponding to removing a chemical element, a compound fragment, or one or more atoms from a compound; a rule or operation corresponding to breaking or removing bonds between atoms of a compound; a rule or operation corresponding to adding or reforming bonds between atoms of a compound; any other rule or any other operation associated with modifying a compound to form another compound; and any other rule or any other operation associated with modifying a compound to form a different compound.

[0069] A set of rules and / or one or more operations can also be selected that conform to any required or necessary structural, physical, and / or chemical constraints associated with the compound, such that any modification of the compound and / or subsequently modified compound is actually feasible. The set of rules and / or one or more operations can also be selected from a set of related chemical groups, which can include one or more of the following: one or more atoms; one or more molecules; one or more other compounds; one or more bonds; one or more functional groups; and / or one or more chemically relevant aspects of a compound. For example, a rule and / or operation can include one or more operations for modifying a specified compound, where the operation can include, by way of example only and not limitation, adding / removing / changing, by way of example only and not limitation, at least one or more of a specific atom, a specific molecule, a specific other compound, a bond, and / or a functional group to the specified compound. The one or more specific atoms, molecules, other compounds, bonds, and / or functional groups can be selected from a set of chemically relevant atoms, molecules, other compounds, and / or compound fragments, bonds, and / or functional groups. These may be predefined and / or selected by an operator, or automatically selected based on a knowledge base, or based on stored lists, or based on one or more collections of data representing chemically related atoms, molecules, other compounds and / or compound fragments, bonds and / or functional groups, etc., for use in modifying a specified compound or subsequent compounds with respect to desired properties.

[0070] For example, multiple n>1 rules / operations for modifying compounds can form a rule set where R iis the i-th rule / operation for modifying the compound, which may include the following rules by way of example only and not limitation:

[0071] R1 - adding a first chemical element to the compound;

[0072] R2 - adding the first compound fragment to the compound;

[0073] R3 - removes the first chemical element from the compound;

[0074] R4—removing the first compound fragment from the compound;

[0075] R5 - adding a second chemical element to the compound;

[0076] R6 - adding a second compound fragment to the compound;

[0077] R7-removes a second chemical element from the compound;

[0078] R8 - removing a second compound fragment from the compound;

[0079] …

[0080] Ri -bonds added or reformed between atoms of a compound;

[0081] Rj-breaks or removes bonds between atoms of a compound;

[0082] …

[0083] R n-1 - any other rules or any other operations relating to the modification of a compound to form another compound; and

[0084] R n - Any other rules or any other operations related to modifying a compound to form a different compound.

[0085] This rule set Each rule / operation can be used one or more times to modify a compound from an initial compound to another compound. One or more rules (R i ), the set of rules can define how to modify a first compound to form another compound based on the ordered sequence of rules. The sequence of rules is used to ensure that the resulting compound can be derived from the ordered sequence of the first compound. There is no guarantee that any ordering of the sequence of rules will produce the same resulting compound. For the purposes of this article, the sequence of rules is considered to be an ordered sequence of rules that should be applied in a specific order to modify the first compound into the compound that would result from following the ordered sequence of rules when modifying the first compound.

[0086] The reinforcement learning techniques according to the present invention described herein may use one or more machine learning techniques or a combination of machine learning techniques to generate one or more candidate compounds. Machine learning techniques may include or represent one or more computational methods or a combination of computational methods that can be used to generate analytical models and algorithms suitable for solving complex problems, such as, by way of example and not limitation, generating or predicting compounds with desired characteristics and / or properties. Machine learning techniques may be used to generate compounds for drug discovery, identification, and / or optimization in the fields of informatics, cheminformatics, and / or bioinformatics.

[0087] The machine learning techniques used to generate candidate compounds from the desired property set and the starting compound may include (by way of example only and not limitation) at least one machine learning technique or combination of machine learning techniques from the following group: recursive neural networks, convolutional neural networks, reinforcement learning algorithms based on neural networks, and any other neural network structures suitable for implementing the present invention described herein. In addition, one or more other neural networks may be applied or used to read / receive compound structures (or molecular structures, etc.) for inputting these compound / molecular structures into the machine learning technique (e.g., reinforcement learning technique) in an appropriate format. Other machine learning techniques may be used to score compounds for the desired property set and / or estimate the properties of compounds for scoring, which may be used to generate one or more or a series of machine learning models designed to predict one or more properties of a compound and output a property value or property score that indicates whether the compound exhibits the property or the degree to which it is associated with the property.

[0088] Other examples of machine learning techniques that may be used by the reinforcement learning techniques according to the present invention as described herein may include or be based on, by way of example only and not limitation, any machine learning technique or algorithm / method that may be trained or adapted to generate one or more candidate compounds, where the training or adaptation of the machine learning technique or algorithm / method is based solely on, by way of example only and not limitation, an initial compound, a set of desired properties of a candidate compound, and / or a set of rules for modifying a compound, which may include one or more supervised machine learning techniques, semi-supervised machine learning techniques, unsupervised machine learning techniques, linear and / or nonlinear machine learning techniques, machine learning techniques associated with classification, machine learning techniques associated with regression, the like, and / or combinations thereof. Some examples of machine learning techniques may include or be based on (by way of example only and not limitation) one or more of active learning, multi-task learning, transfer learning, neural message parsing, one-shot learning, dimensionality reduction, decision tree learning, association rule learning, similarity learning, data mining algorithms / methods, artificial neural networks, deep neural networks, deep learning, deep learning neural networks, inductive logic programming, support vector machines, sparse dictionary learning, clustering, Bayesian networks, representation learning, similarity and metric learning, sparse dictionary learning, genetic algorithms, rule-based machine learning, learning classifier systems, and / or one or more combinations thereof, etc.

[0089] Some examples of supervised machine learning techniques may include or be based on, by way of example only and not limitation, artificial neural networks, deep neural networks, association rule learning algorithms, a priori algorithms, Algorithms, example-based reasoning, Gaussian process regression, gene expression programming, group methods for data processing (GMDH), inductive logic programming, example-based learning, lazy learning, learning automata, learning vector quantization, logistic model trees, minimum message length (decision trees, decision diagrams, etc.), nearest neighbor algorithms, simulation modeling, probably approximately correct (PAC) learning, chain descent rules, knowledge acquisition methods, symbolic machine learning algorithms, support vector machines, random forests, classifier ensembles, bootstrap aggregation (BAGGING), boosting algorithms (meta-algorithms), ordered classification, fuzzy networks (IFNs), conditional random fields, analysis of variance, quadratic classifiers, k-nearest neighbors, boosting algorithms, sprint algorithms, Bayesian networks, naive Bayes, hidden Markov models (HMMs), hierarchical hidden Markov models (HHMMs), and any other machine learning technique or machine learning task that can infer functions or generate models from labeled and / or unlabeled training data.

[0090] Some examples of unsupervised machine learning techniques may include or be based on (by way of example only and not limitation) the expectation maximization (EM) algorithm, vector quantization, generative topography, the information bottleneck (IB) method, and any other machine learning technique or machine learning task that is capable of inferring a function describing a hidden structure, and / or generating a model from unlabeled data, and / or by ignoring labels in a labeled training dataset, etc. Some examples of semi-supervised machine learning techniques may include or be based on (by way of example only and not limitation) active learning, generative models, low-density separation, graph-based methods, joint training, transduction, or any other machine learning technique, task, or a class of unsupervised machine learning techniques that is capable of training with unlabeled datasets and / or labeled datasets, etc.

[0091] Some examples of artificial neural network machine learning techniques may include or be based on, by way of example only and not limitation, artificial neural networks, feedforward neural networks, recurrent neural networks, convolutional neural networks, autoencoder neural networks, extreme learning machines, logic learning machines, self-organizing maps, and other artificial neural network machine learning techniques or connectivity systems / computing systems inspired by biological neural networks that constitute animal brains. Some examples of deep learning machine learning techniques may include or be based on, by way of example only and not limitation, deep belief networks, deep Boltzmann machines, deep neural networks, deep convolutional neural networks, deep recurrent neural networks, hierarchical temporary memories, deep Boltzmann machines (DBMs), stacked autoencoders, and / or one or more of any other machine learning techniques.

[0092] Although reinforcement learning technology can also be part of the class of machine learning technology, those skilled in the art will understand that the reinforcement learning technology herein can use or apply any one or more suitable machine learning technologies or combinations thereof, and as mentioned / described above, generate one or more candidate compounds based on the initial compound (or subsequent candidate compound) and the desired properties that the resulting candidate compound is required to exhibit.

[0093] Figure 1a 1 is a flow chart illustrating an exemplary process 100 for designing compounds that exhibit one or more desired properties according to the present invention. Process 100 can use machine learning techniques and a set of desired properties of candidate compounds to generate candidate compounds, which can be scored and then, if necessary, used to adjust or update the machine learning technique so that the machine learning technique generates further improved candidate compounds that can better exhibit the desired properties. The desired properties are predetermined or input to process 100 at the outset and can include data representing a series of characteristics or properties that the candidate compound is expected to exhibit. The steps of process 100 can include one or more of the following steps:

[0094] In step 102, data representing the initial compound (e.g., a starting compound, or a molecule, or a set of compound fragments) is input to a machine learning technique that is used to modify the initial compound and generate a candidate compound (or one or more candidate compounds). Data representing the desired properties of the candidate compound are also input to the process, which can be used for machine learning techniques and / or scoring any candidate compounds generated. A set of rules for modifying the compound can also be input to process 100 to allow the machine learning technique to select a sequence of rules or generate a sequence of operations for modifying the compound to generate a candidate compound. The machine learning technique can also be configured to learn or be trained to model the rules / operations for modifying the compound.

[0095] In step 104, a machine learning technique is used to generate a candidate compound or one or more candidate compounds by modifying the first compound (e.g., the initial compound and / or any subsequent candidate compounds) based on the desired properties and the set of rules for modifying the compound. The machine learning technique outputs one or more candidate compounds.

[0096] In step 106, the candidate compounds are evaluated and scored for the desired properties. One or more scores can be associated with each desired property. Thus, each candidate compound can be scored for each desired property. Atomistic computer simulations can be used to simulate the candidate compounds to determine a measure of each desired property, which can be used to generate a score for each desired property. The scores for each desired property can then be weighted and / or summed to generate an overall score that indicates how close the candidate compound is to exhibiting the desired property.

[0097] In step 108, a determination is made as to whether additional candidate compounds are needed based on the scores from step 106. This determination can use the previous scores from the previous iteration of process 100 to determine whether the scores are a significant improvement over the candidate compounds previously generated by the machine learning technique. If the scores indicate that additional compounds are needed (e.g., "yes"), the machine learning technique used to generate the candidate compounds can proceed to step 110, where the machine learning technique can be adjusted or updated based on the scores (e.g., one or more scores associated with each desired property, or a weighted / aggregate score). The machine learning technique can be updated or adjusted before starting another iteration of generating / modifying compounds. If it is determined that additional candidate compounds are not needed, for example, if it is determined that one or more candidate compounds exhibit the desired properties, the process proceeds to step 112. This determination may also include one or more further determinations or an overall determination based on, by way of example only and not limitation, one or more of the following: whether a maximum number of iterations have occurred; whether there are no further significant improvements in the candidate compounds compared to the previous iteration of the candidate compounds; whether the scores for each candidate compound are stable compared to the previous iteration; whether the scores indicate that one or more candidate compounds exhibit the desired property; and / or there are no further significant improvements in the candidate compounds, and it is clear that these candidate compounds are the best compounds that can be achieved.

[0098] In step 110, the machine learning technique is adjusted or updated based on the scores and / or determinations. The overall score for each candidate compound may be based on one or more scores associated with each desired property. The score indicates the extent to which the candidate compound matches or fits the desired property. The reinforcement learning technique initiates an update or adjustment of the machine learning technique based on the score, which involves updating the parameters, coefficients, and / or weights of the machine learning technique. The reinforcement learning technique may penalize the machine learning technique based solely on, but not limited to, one or more of the following conditions: the score indicates that the properties of the candidate compound are further away from the desired property; the modified molecule / compound is too large / too small; the modified molecule / compound exhibits any other undesirable properties or differences from the desired compound and / or desired properties. On the other hand, the reinforcement learning technique may reward the machine learning technique based on, by way of example only and not limitation, one or more of the following: the score indicates that the modified molecule exhibits properties that are closer to the desired property.

[0099] Although the machine learning technology is adjusted or updated after step 110, it will be understood by those skilled in the art that the machine learning technology can be adjusted or updated at any time after the scores of the candidate compounds in step 106 have been output or known. In any case, the machine learning technology can be adjusted or updated before repeating the generation step 104 of process 100. This is performed before performing another iteration to generate another one or more candidate compounds. In addition, the update step 110 can include a further determination as to whether the machine learning technology needs to be updated based on the scores. For example, it may be advantageous not to update the machine learning technology immediately and allow it to perform another iteration or generate another candidate compound. Instead of interfering with the machine learning technology by updating the machine learning technology to encourage it to generate "better" candidate compounds, the machine learning technology can be interfered with by allowing it to generate one or more additional candidate compounds based on modifying the current one or more candidate compounds. In addition, the update step 110 can also include a determination as to whether the machine learning technology should revert to a previous version of the machine learning technology, at which time a candidate compound with a higher score than the current candidate compound may have been generated. The previous version of the machine learning technique may be used, along with the corresponding previous candidate compounds, to perform another iteration or generation of additional candidate compounds that may be evaluated and / or scored in step 106 and / or to perform a decision to continue with the machine learning technique iteration / update in step 108 .

[0100] After step 110, the reinforcement learning technique performs another iteration of steps 104 and 106 of the design process 100. An iteration counter may be updated to track the number of iterations that have been performed. In the next iteration, the process 100 proceeds to step 104, where the machine learning technique, which may have been updated in step 110, is used to generate one or more candidate compounds from, by way of example only and not limitation, either: a) the initial compound used as input in step 102; or b) one of the candidate compounds from the previous iteration. The machine learning technique applies another sequence of rules / operations to modify the initial compound and output another modified candidate molecule / compound for scoring in step 106.

[0101] In step 112, it has been determined in step 108 that process 100 should cease iterations and output data representing one or more candidate compounds. The data representing each candidate compound may include, by way of example only and not limitation, a description or representation of the candidate compound, a property score and / or an overall score assigned to the candidate compound, the sequence of rules / operations performed by the machine learning technique to modify the starting candidate compound and ultimately obtain the candidate compound, and any other information that may be useful to the expert in evaluating whether the output candidate compound is feasible given how closely the output candidate compound exhibits the desired properties input in step 102.

[0102] In addition, all generated candidate compounds and corresponding scores can be stored, and when it is decided to output one or more candidate compounds, the stored generated candidate compounds can be ranked based on their scores, and the top k rankings (k ≥ 1) and data representing the candidate compounds can be output. In addition to outputting the candidate compounds, data representing the rule sequence used to generate each candidate compound can also be stored and output to provide evidence about how the candidate compound is composed and / or why it exhibits or is close to exhibiting the desired properties.

[0103] Inputting compounds to the machine learning technique in step 102 and / or step 104 may include, by way of example only and not limitation, inputting: an initial compound; an input compound set; an input compound fragment set that can be combined or modified to generate one or more candidate compounds; an input compound or compounds generated by a previous iteration of process 100; or any one or more of these inputs or a combination thereof. The machine learning technique may be configured to receive a compound or one or more compounds / fragments and generate a candidate compound or a set of candidate compounds based on a desired property and a set of rules and / or operations for modifying the compound.

[0104] Further modifications to process 100 may include: in step 108, determining whether to repeat generating step 104 and / or updating the machine learning technique in step 110 may be based on, by way of example only and not limitation: a score indicating that the candidate compound is closer to a compound exhibiting the desired property; a score indicating that the second compound exhibits the desired property; whether a predetermined number of iterations of repeating generating step 104 have been achieved or satisfied; and the second compound exhibiting at least one or more of the desired properties, whether or not any further improvements to the second compound are possible.

[0105] As described in step 106 of method 100, candidate compounds are evaluated and scored for the desired properties. The scoring of each desired property can be based on (for example only but not limited to) a certainty score, wherein when one or more candidate compounds substantially exhibit most or all of the desired properties, one or more candidate compounds have a positive certainty score. When one or more candidate compounds substantially do not exhibit some or even any of the one or more desired properties, the certainty score can be a negative certainty score. When a candidate compound substantially exhibits some or most of one or more desired properties (including but not limited to), the candidate compound can also have a certainty score between a positive certainty score and a negative certainty score. The certainty score can be expressed as a percentage certainty score, wherein the maximum positive certainty score is 100% and the minimum negative certainty score is 0%, wherein the certainty score can be between these two extreme values.

[0106] Evaluation and scoring of candidate compounds based on a set of desired properties may include analyzing the candidate compound for each of the desired properties, then calculating a property value or score for each desired property associated with the candidate compound, and / or calculating an aggregate score for the candidate compound based on the analysis. Scoring of the candidate compound may include calculating one or more property score metrics / values / scores based on, by way of example only and not limitation, machine learning models for properties such as solubility, toxicity, drug-target interaction, bioavailability, metabolism, etc., which may be configured to predict compound property values / scores associated with the properties, or calculating property values / scores for the candidate compound based on computer simulations and / or from laboratory experiments or tests in a laboratory setting.

[0107] For example, the analysis may include performing computer simulations on the candidate compound associated with one or more desired properties. Alternatively or additionally, the analysis may also include using a knowledge-based expert system to analyze the candidate compound to determine whether the candidate compound exhibits or is closer to exhibiting one or more desired properties or all desired properties. Alternatively or additionally, although it may be more expensive and take longer, the analysis may include using laboratory experiments to analyze the candidate compound to determine whether the candidate compound exhibits or is closer to exhibiting one or more desired properties or all desired properties in the desired property set.

[0108] Alternatively or additionally, the evaluation and / or scoring of candidate compounds can be based on one or more machine learning techniques, each of which may have been trained based on known compounds or labeled training data to predict whether a compound has a particular characteristic or property. Each trained machine learning technique can be associated with a different characteristic or property of a compound and can be configured to output a property value / score, such as, but not limited to, a probability, a certainty score, or other classification or outcome indicating whether the input compound has the particular characteristic or property that the machine learning technique was trained to detect / determine.

[0109] Thus, a candidate compound can be input to one or more machine learning techniques associated with a set of desired properties, where each machine learning technique outputs, for example, a property value / score, such as a certainty score associated with a particular desired property, to generate a set of probability / certainty scores that can be used to determine whether the candidate compound is closer to exhibiting the desired property than in a previous iteration of process 100. Alternatively or additionally, the set of probability or certainty scores can be weighted and / or aggregated to provide an overall probability or certainty score indicating that the candidate compound is closer to exhibiting the set of desired properties than in a previous iteration of process 100.

[0110] Although the scoring of candidate compounds is described, by way of example and not limitation, as being performed using atomistic computer simulations and / or atomistic computer models, and / or using a machine learning model that simulates or measures one or more properties of a compound and that is trained to predict whether a compound exhibits or is associated with a particular property, one skilled in the art will appreciate that the scoring of candidate compounds can be performed using another machine learning technique that is trained to score candidate compounds for a range of desired properties. Furthermore, the scoring machine learning technique can be based on a reinforcement learning technique that receives a candidate compound, receives a desired property, and iteratively learns how to score the candidate compound over time.

[0111] While the scope of machine learning techniques has been described herein, it is preferred that the machine learning techniques that may be used may include, by way of example only and not limitation, at least one machine learning technique or combination of machine learning techniques from the following group: a recursive neural network configured to predict a second compound exhibiting a desired set of properties starting from a first compound; a convolutional neural network configured to predict a second compound exhibiting a desired set of properties starting from a first compound; a reinforcement learning algorithm configured to predict a second compound exhibiting a desired set of properties starting from a first compound; and any neural network structure configured to predict a second compound exhibiting a desired set of properties starting from a first compound. Once the hidden layer structure of the neural network is determined, the weights, parameters, and coefficients may be updated and / or changed based on various update algorithms (e.g., backpropagation algorithms, etc.) based on the scores used in process 100.

[0112] Figure 1b 1 is a flow chart illustrating an exemplary apparatus 120 for designing compounds that exhibit one or more desired properties according to the present invention. Apparatus 120 may include an iterative loop comprising a machine learning device 122, a scoring device 124, and a determination / update device 126, which may be used to iteratively generate one or more sets of candidate compounds. Output device 128 is configured to output the candidate compounds and the set or sequence of rules used to generate the output candidate compounds when determination / update device 126 determines that further iterations are not necessary or that an improved set of candidate compounds that exhibit or more closely resemble the desired properties will be generated.

[0113] The machine learning device 122 can be based on any suitable machine learning technology that can generate candidate compounds based on an input compound or input compound set, a desired property, and / or a rule set for modifying the input compound. The machine learning device 122 can output a candidate compound set (e.g., one or more candidate compounds) to a scoring device 124. The scoring device 124 is configured to evaluate and / or score the candidate compound set based on the desired property. A score set for each property and / or each candidate compound can be output, or a total score set for each candidate compound (or a total property score based on a desired property set) can be output from the scoring device 124. These candidate compound scores can be used to determine whether one or more candidate compounds are closer to exhibiting the desired property, and whether the device 120 should perform another iteration to generate other candidate compounds. The candidate compound scores of each candidate compound can be used to select those candidate compounds that are more likely to generate other candidate compounds that are closer to exhibiting the desired property.

[0114] The determination / update device 126 receives the scores for the candidate compounds from the scoring device 124 and can use previous or historical performance / scoring data to determine whether the machine learning technique device 122 has improved in outputting candidate compounds that more closely resemble compounds exhibiting all desired properties, or will improve in the future to output candidate compounds accordingly. The scores can also be used to trigger whether the machine learning technique device 122 updates or further adjusts the machine learning technique. The machine learning technique device 122 can use the scores to adjust the relevant weights, parameters, and / or coefficients of the machine learning technique. The determination device 126 can use the scores to reward or penalize the machine learning technique during the update, which will help it make different decisions or rule selections when modifying other candidate compounds. The determination device 126 can also consider whether the machine learning technique needs to be reverted to a previous state, thereby updating the machine learning technique with the previous state of the machine learning technique. Once the determination device 126 determines that repeated generation of candidate compounds has been triggered, after updating / reverting the machine learning technique, the machine learning technique device 122 can be reconfigured to generate additional candidate compounds based on (by way of example only, but not limitation) the initial compound or set of compounds, the current candidate compound, and / or one or more previous candidate compounds.

[0115] The determination / update device 126 can determine or determine that no further iterations are needed to generate additional candidate compounds, and therefore output the candidate compound that has been found to be the best one or more candidate compounds that exhibit the desired properties based on the scores. That is, all generated candidate compounds can be stored and ranked based on their scores, and the top k rankings (k ≥ 1) and data representing the candidate compounds can be output. In addition to outputting the candidate compounds, data representing the regular sequence used to generate each candidate compound can also be stored and output to provide evidence about how the candidate compound is composed and / or why it exhibits or comes close to exhibiting the desired properties.

[0116] Figure 2a 2 is a flow chart illustrating an exemplary process 200 followed by a machine learning technique when generating candidate compounds that may exhibit one or more desired properties according to the present invention. The exemplary process 200 may be implemented in step 104 of the reinforcement learning technique process 100, which may be iteratively used to generate one or more sets of candidate compounds. As described in steps 108 and 110 of the process 100, the machine learning technique may be adjusted and updated based on the scores (e.g., rewards and / or penalties) of any output candidate compounds. In this way, the machine learning technique "learns" to select an improved set or sequence of rules for modifying compounds to generate candidate compounds that are most likely to be closer to the set exhibiting the desired properties. The steps of the machine learning technique process 200 may be (by way of example only and not limitation) as follows:

[0117] In step 202, the machine learning technology receives data representing a set of one or more compounds, data representing a set of rules, and / or data representing candidate compounds that can be modified to exhibit a set of desired properties. The rule set may include a set of operations for modifying compounds. In step 204, the machine learning technology may select one or more rules from the rule set for modifying the set of one or more compounds and generate a set of one or more candidate compounds. For example, the machine learning technology may receive compounds and then generate a first set of candidate compounds, each candidate compound being modified based on the received compound using a different rule from the rule set. The machine learning technology may be configured to estimate, evaluate, and / or score each candidate compound in the set of candidate compounds based on the set of desired properties. This may help the machine learning technology select the best candidate compound and the corresponding rule used to generate each best candidate compound from the first set of candidate compounds. That is, the machine learning technology may rank the set of candidate compounds and corresponding rules based on the estimates, evaluations, and / or scores, and then generate a second set of candidate compounds based on the highest-ranked candidate compound from the first set of candidate compounds and the corresponding rule set. The machine learning technology may perform this iteration multiple or a set number of times before moving to step 206. In step 206, the machine learning technique may output a set of candidate compounds and / or a set of corresponding rules for modifying the received compound. Each candidate compound outputted may have a corresponding set of selected rules that may be used to generate the output candidate compound from the input compound or even the initial compound inputted to the machine learning technique.

[0118] Figure 2b 2 is a schematic diagram of an exemplary apparatus 210 illustrating a machine learning technique 212 that may be used in an iterative loop of apparatus 120 and / or process 100. The machine learning technique 212 receives data representing one or more input compounds 214 and is intended to generate a candidate compound or one or more candidate compounds 220, each of which may exhibit a closer approximation to exhibiting one or more desired properties 218 than the one or more input compounds 214. The machine learning technique 212 may be based on any machine learning technique described herein or a combination thereof. The machine learning technique 212 may receive data representing at least a compound 214 or one or more compounds (or compound fragments), and the machine learning technique 212 may use this data as a basis, along with a set of rules 216 for modifying compounds, to generate a set of candidate compounds 220 that are more likely to exhibit a desired property 218 than the input compounds.

[0119] However, it should be noted that in early iterations of the device 120, the machine learning technology 212 may initially generate candidate compounds that are unlikely to exhibit the desired property 218. That is, when scoring the candidate compounds, the candidate compounds may underperform, and therefore, based on the scores, the machine learning technology 212 may receive triggers or updated triggers, as well as scores that penalize them during updates / adjustments of the machine learning technology 212. This serves to teach the machine learning technology 212 not to select those rule sequences; or not to initially select a portion of the rule set for modifying a compound; and / or not to make decisions that result in such underperforming candidate compounds. Thus, the device 200 becomes part of a reinforcement learning technology, receiving updates / adjustments, and rewards or penalties based on the candidate compounds it outputs. Over time, or over multiple iterations of the device 100's reinforcement learning loop, the machine learning technology 212 of the device 200 may be trained to select one or more rule sequences that can be used to generate candidate compounds that more closely exhibit the desired property, or indeed exhibit all of the desired properties with a high degree of certainty.

[0120] Figure 3a 304a-304n, 304a-304n, 310-312 and a plurality of edges R1, ..., R2, R3, R4, R5, R6, R7, R8, R9, R10, R11, R12, R13, R14, R15, R16, R17, R18, R19, R20, R21, R22, R23, R24, R25, R26, R27, R28, R29, R29, R30, R31, R32, R33, R34, R35, R36, R37, R38, R39, R31, R31, R32, R33, R34 ...5, R36, R37 n The multiple nodes include a root node 302, the root node 302 represents a compound, and multiple edges R1, ..., R n From this compound are connected the child node sets 305a-305n. n = represents a set of rules for modifying compounds. Each rule in the rule set corresponds to an edge. Therefore, child nodes 305a-305n are connected to each edge R1, ..., R from the root node 302. n , where each child node 305a-305n represents the corresponding rule R1,…,R n (Indicates a connecting edge) The modified compound of the root node 302 .

[0121] The tree 300 can be grown down to m levels, where each node at each level uses the same edge set R1, ..., R n That is, each edge R1,…,R n A parent node 302 is connected to child nodes 304a-304n, where the parent node 302 represents a compound, and each edge R1, ..., R nRepresents rules R1,…,R n A rule (or operation) in a set of (or multiple operations) is executed on the compound of the parent node 302 to obtain the compounds of the child nodes 304a-304n. The root node 302 of the tree 300 is the first compound that can be input into the machine learning technique, and the subsequent nodes 304a-304n, 304a-304n, 304a-304n, 310 and 312 correspond to one or more candidate compound sets or multiple candidate compounds. Assume that each node in the tree 300 uses the same edges R1, ..., R n If there is a set of rules along all the rule edges, then following the path along the edge down the tree to a particular node at level m (e.g., Lm) will give a sequence of rules that can be used to modify the first compound into a compound associated with the particular node at level m. Although parsing the entire tree 300 based on the rule set along all the rule edges can generate all possible candidate compounds and the corresponding sequence of rules for each compound, this will result in exponential complexity for machine learning techniques. For example, if there are n rules in the rule set for modifying a compound, then the mth level (m≥0) of the entire tree 300 will have many nodes or candidate compounds.

[0122] Thus, instead of generating the entire tree 300 up to the mth level, machine learning techniques can expand the tree data structure based on evaluating or scoring each of one or more nodes corresponding to the set of candidate compounds. Alternatively, a tree search can be performed on the tree data structure 300 to generate a set of candidate compounds based only on those rule sequences that generate candidate compounds with the best scores or that more closely approximate a desired set of properties.

[0123] For example, a simple machine learning technique can be configured to maintain a tree-like data structure 300 and increase it only one level at a time at each iteration of process 100. Thus, a first compound is input as the root node 302 of the tree, and in a first iteration of process 100, the machine learning technique can output a set of candidate compounds based on the set of child nodes 304a-304n in step 104. In step 106, process 100 can evaluate the set of candidate compounds and output a corresponding set of scores. In step 108, when there are too many candidate compounds, a decision can be made to perform another iteration, so that the machine learning technique can be updated based on the set of scores. For example, those child nodes 304a-304n with scores above a certain threshold can be retained, while those child nodes 304a-304n with scores below a certain threshold can be pruned or removed. Thus, in the next iteration, the machine learning technique can maintain and / or grow the tree-like structure 300 based only on those nodes 304a-304n that have been retained and the set of rules in each iteration. However, assume that the rule sets R1, ..., R ncan be very large (e.g., greater than 1000 or greater than 1,000,000), which still generates a very large set of candidate compounds to be selected from in each iteration. Therefore, the machine learning technique can be configured to estimate, evaluate and / or select only those regular edges R1, ..., R2 at each level of the tree data structure 300 that are more likely to generate suitable candidate compounds. n and / or child nodes.

[0124] Figure 3b 3 is a flow chart of an example process 320 for implementing a tree-based machine learning technique according to the present invention. The process 320 can be implemented in each iteration of the process 100 using a machine learning technique, wherein the machine learning technique is configured to maintain a tree-based data structure. The process can include the following steps: In step 322, a first compound can be input as a root node, or a previously generated tree can be input, wherein each leaf node represents a set of candidate compounds. In step 324, based on the rule set R1, ..., R n , and also based on machine learning technology, to generate the child nodes of the current root node and / or parent node (or leaf node), the machine learning technology can be configured to select the rule set R1, ..., R n to generate a subset of child nodes. In step 326, the machine learning technique may be further configured to select the newly created child node set as the candidate compound set. The selection may be configured and based on the implementation of the machine learning technique, as well as based on whether it is updated or how it is updated in each iteration of process 100. In step 328, it is determined whether to continue parsing the tree data structure and move down to the next level in the tree and generate other child nodes, or output the selected child node set as the selected candidate compound set. If it is determined to parse the tree data structure (e.g., "yes"), then in step 330, the next level of the tree data structure is generated, and step 324 is executed to generate other child nodes of this level based on the rule set and the machine learning technique. If it is determined to output the candidate compound set (e.g., "no"), then in step 332, the machine learning technique outputs the candidate compound based on the selected child node set. The machine learning technique can also output a corresponding rule sequence, which can be generated by following the path from the root node to each selected child node.

[0125] Note that the machine learning technology may be updated in each iteration of process 100, and the machine learning technology may be triggered, starting with the first compound as the root node, using the updated machine learning technology to (by way of example only and not limitation) restart the generation and maintenance of the tree data structure, iteratively generate child nodes and parse the tree data structure up to multiple levels or multiple iterations, select and output a set of child nodes as candidate compounds; and / or starting with the current candidate set or the currently parsed tree data structure, continue to generate child nodes based on the updated machine learning technology; or starting with a previous set of candidate nodes, using a previously parsed tree data structure, and continuing to parse the previous tree data structure based on the updated machine learning technology, which should generate other different subsets of child nodes.

[0126] Figure 3c and 3d is a schematic diagram of exemplary tree data structures 340 and 350 according to the present invention, which have been based on Figure 3b The process 320 of the embodiment parses the exemplary tree data structures 340 and 350 to generate a set of one or more candidate compounds that may be closer to exhibiting the desired properties. Assume that the process 100 uses a tree-based machine learning technique based on the process 320. The tree-based machine learning technique receives compounds and a set of rules R1, ..., R n In the first iteration of process 100, the machine learning technique constructs a tree 340 starting from the root node 302, which represents the initial compound. The machine learning technique then constructs a tree 340 based on the rule set R1, ..., R n The root node 302 is expanded to generate one or more child nodes 304a-304n. The structure of the machine learning technology can be configured to select or evaluate which child nodes 305a-305n are suitable or retained as candidate compounds. Figure 3c As shown, the tree-based machine learning technique retains only child node 304a as a candidate compound and discards child nodes 304b-304n due to some reasons of the current structure / weights / parameters or other data representing the underlying machine learning technique.

[0127] The machine learning technique may decide (e.g., in step 328 of process 320) to generate another set of child nodes based on the retained child node 304a. In this case, the structure of the machine learning technique may be configured to select or evaluate which child nodes 306a-306n may be suitable or retained as additional candidate compounds. Figure 3cAs shown, the tree-based machine learning technique retains only child node 306b as a candidate compound for some reason and discards child nodes 306a and 306c-306n. At this point, the machine learning technique may decide (e.g., in step 328 of process 320) to output the final child node set as a candidate compound set and move to step 332 to output child node 306b as a candidate compound. Based on applying rule sequences R1 and R2 to the first compound, the candidate compound is generated from the first compound (e.g., root node 302). That is, R1 is applied to the first compound represented by root node 302 to generate a compound represented by child node 304a, and the next rule R2 in the sequence is applied to the compound represented by child node 304a to generate a compound represented by child node 304b. Therefore, a candidate compound based on the rule sequence {R1, R2} can be output by machine learning technology.

[0128] In step 106 of process 100, the candidate compounds output by the tree-based machine learning technique can be evaluated and scored. Assuming this is the first iteration of process 100, it is most likely that step 108 determines that additional candidate compounds are needed. In step 110, a suitable update algorithm can be used to update the current structure / weights / parameters based on the scores of the output candidate compounds, or to update other data representing the underlying tree-based machine learning technique (for example, if the underlying machine learning technique is based on a neural network, a weight update based on a backpropagation technique can be used). Assuming that the machine learning technique has been updated, in this example, the machine learning technique begins to rebuild the tree based on the starting compound at the beginning. This is because, in view of the update to the machine learning technique, the machine learning technique can make different decisions / selections for the child nodes to potentially generate improved candidate compounds.

[0129] Now refer to Figure 3d After the tree-based machine learning technique is updated in step 104, the updated machine learning technique receives the compound (original compound) and the rule set R1, ..., R n In the second iteration of process 100, the machine learning technique rebuilds the tree 350 starting from the root node 302, which again represents the initial compound. The machine learning technique then generates a tree 350 based on the rule set R1, ..., R n The root node 302 is expanded to generate one or more child nodes 304a-304m and 304n. The structure of the newer machine learning technology may have been configured to select or evaluate which child nodes 304a-304m and 304n may be suitable or retained as candidate compounds. Figure 3dAs shown, due to the new updated structure / weights / parameters, or due to other data representing the underlying machine learning technology, the updated tree-based machine learning technology only retains child node 304m as a candidate compound and discards other child nodes 304a-304n except child node 304m.

[0130] The updated machine learning technique may further determine (e.g., in step 328 of process 320) based on the retained child node 304m and the rule set R1, ..., R n To generate other sets of child nodes 350a-350n. In this case, the structure of the machine learning technology can be configured to select or evaluate which child nodes 350a-350n may be suitable or retained as other candidate compounds. Figure 3d As shown, for some reason, the tree-based machine learning technique retains only child node 352a as a candidate compound and discards child nodes 352b-352n. At this point, the machine learning technique may decide (e.g., in step 328 of process 320) whether to output the final set of child nodes as the set of candidate compounds. In this case, the machine learning technique continues to select the candidate compound based on child node 352a and the rule set R1, ..., R n Generate other sets of child nodes 354a-354n. In this case, the structure of the machine learning technology can be configured to select or evaluate which child nodes 354a to 354n are suitable or retained as other candidate compounds. Figure 3d As shown, the tree-based machine learning technique retains only child node 354n as a candidate compound and discards child nodes 354a-354m for some reason. At this point, the machine learning technique may decide (e.g., in step 328 of process 320) to output the final set of child nodes as a set of candidate compounds.

[0131] In this example, the process 320 proceeds to step 332 to output the child node 354n as a candidate compound. n-1 , R1 and R n Applied to the first compound, the candidate compound is generated from the first compound (e.g., root node 302). n-1 is applied to the first compound represented by root node 302, generating the compound represented by child node 304m, applying the next rule R1 in the sequence to the compound represented by child node 304m, generating the compound represented by child node 352a, and applying the next rule R n Applied to the compound represented by child node 352a, a compound represented by child node 354n is generated. Therefore, a rule sequence (R n-1 , R1, R n ) candidate compounds.

[0132] In step 106 of process 100, the rule sequence (R n-1 , R1, R n ) to evaluate and score the candidate compounds. Assuming this is the second iteration of process 100, it is most likely that step 108 determines that additional candidate compounds are needed. In step 110, a suitable update algorithm can be used to update the current structure / weights / parameters based on the output candidate compound scores, or update other data representing the underlying tree-based machine learning technology (for example, if the underlying machine learning technology is based on a neural network, a weight update based on backpropagation technology can be used). Assuming that the machine learning technology has been updated or based on the scores, and assuming that the scores of the candidate compounds in this iteration of process 100 are better than the previous scores, the machine learning technology can start to rebuild the tree based on the initial starting compound; or if the candidate compound is closer to a compound that exhibits the desired properties, the current tree-based structure can be used to further parse or generate other child nodes on the tree 350. In any case, given the updates to the machine learning technology, the machine learning technology should make different decisions / selections on the child nodes and can generate further improved candidate compounds. The current or rebuilt tree and the starting compounds described herein can be further iterated in processes 100 and 320.

[0133] Although a tree-based data structure can be used to store and / or generate a candidate compound set, an alternative exemplary compound modification strategy based on one or more machine learning techniques can also be applied. For example, a machine learning technique that generates an N-dimensional operation / rule space based on a rule set can be used, where N is large enough so that each rule / operation in the rule / operation set can be represented by a unique N-dimensional vector in an N-dimensional vector space (e.g., N>>2). The elements of each N-dimensional vector can be (as an example but not limited to) real values ​​and / or continuous values. An exemplary machine learning technique that can learn and generate an N-dimensional vector space based on the rule / operation set can include (as an example but not limited to) a structure based on a neural network. For example, a neural network structure can use one or more hidden layers of hidden units or hidden nodes that can be trained to generate an N-dimensional vector space for a rule set, and then the rule / operation set can be used to select an appropriate rule for modifying a compound.

[0134] The rule / operation set for modifying compound has a large amount of possible rules / operations that can be made for modifying compound.For example, in the rule set for modifying compound, the possibility of 1000 or 1,000,000 different rules / operations can be present. Although the data structure based on tree may be useful, when the quantity of rule / operation greatly increases and may become more inefficient, the data structure based on tree is then restricted. Instead, the rule set can be encoded or mapped to the machine learning technique in N-dimensional space (so-called N-dimensional rule / operation vector space) to help select the rule sequence for modifying compound, to generate the suitable candidate compound set that can show required property. Compound can be mapped in N-dimensional rule / operation vector space, then the rule / operation vector closest to the mapped compound or almost closest to it can be selected to modify the compound.

[0135] This encoding / mapping can be performed by a variety of machine learning techniques, such as (by way of example only and not limitation) a neural network structure. A neural network structure typically uses a hidden layer that can be configured to encode / map a set of rules into an N-dimensional space, wherein each rule in the set of rules is encoded or represented as a rule / action vector in an N-dimensional vector space. The neural network can also be configured to map the starting compound into the N-dimensional rule / action space as a compound prototype rule / action vector. The machine learning technique can then search for or determine one or more rule / action vectors that are closest to the compound prototype rule / action vector. The compound can then be modified using the rules / actions associated with the one or more closest rule / action vectors determined to generate one or more corresponding candidate compounds.

[0136] For example, a nearest neighbor search algorithm can be used to determine one or more rules / operation vectors closest to the mapped compound in an N-dimensional vector space (so-called N-dimensional rule / operation vector space). There are many applicable nearest neighbor algorithms, such as, by way of example only and not limitation, k nearest neighbor algorithms, approximate nearest neighbor algorithms, all nearest neighbor algorithms, etc. For simplicity, it is assumed that the coding machine learning technology can be configured to solve the k-nearest neighbor search problem and find the k nearest neighbor rules / operation points closest to the mapped prototype operation / rule in the N-dimensional space, where k≥1. If k>1, multiple candidate compounds can be generated. Once k nearest neighbor operations / rule sets have been identified in the N-dimensional space, they can be decoded or demapped to the corresponding operations / rules of the rule set, and applied to the corresponding compound to modify the compound and generate one or more candidate compounds.

[0137] Therefore, the encoding machine learning technology can encode all compounds and encode all operations / rules that can be performed on the compounds into an N-dimensional space, and determine the k nearest neighbor operations / rules for the compounds. Therefore, the compounds and operations / rules are located in the same N-dimensional space. This N-dimensional space defines all possible rules / operations that can be taken on the compounds. By mapping the compounds into the N-dimensional space as prototype rules / operations, or as so-called compound prototype rule / operation vectors, it is possible to predict the most likely rules / operations that should be taken to modify the compounds into candidate compounds that may be closer to compounds that exhibit the desired properties. For example, the nearest neighbor rule / operation vector in the N-dimensional space that is closest to the compound prototype / rule vector can be used to generate a set of candidate compounds.

[0138] Applying the encoded machine learning technique to the reinforcement learning technique of process 100 can allow the encoded machine learning technique to adjust the N-dimensional space so that it selects a rule sequence from the rule set that can generate a set of one or more candidate compounds that can more closely resemble, learn, or exhibit the desired properties. At the beginning, the neural network of the machine learning technique can be uninitialized and the possible rules / operations that should be applied to the compounds are unknown. Instead, the neural network can simply generate the N-dimensional space based on the set of rules and compounds, and then when process 100 is repeated, the neural network of the machine learning technique can be updated based on the scores (e.g., in steps 106 to 110). The scores indicate how close one or more candidate compounds are to the compounds that exhibit the desired properties and can be used to update the neural network, thereby improving the N-dimensional vector space.

[0139] That is, the neural network can be rewarded if the candidate compound generated by the neural network is closer to the compound exhibiting the desired property, or the neural network can be penalized if the candidate compound generated by the neural network exhibits a property that is not as good as the desired property. When the neural network is updated (or the machine learning technique is encoded) based on the scores of process 100, the neural network will improve the N-dimensional space, and therefore, the rule / operation point that defines the candidate compound that is closer to the desired property and the prototype rule / operation will move closer together in the N-dimensional space.

[0140] As process 100 updates the neural network based on the scores, where the scores are based on whether the candidate compound is getting closer to the desired property, the N-dimensional space represented by the network or by the neural network reflects how close the candidate compound is getting to the desired property, and ultimately what the candidate compound is likely to look like.

[0141] Figure 4a4 is a flow chart illustrating an exemplary encoding process 400 for generating compounds exhibiting one or more desired properties based on machine learning techniques for encoding spaces according to the present invention. The process 400 can also be implemented using machine learning techniques in each iteration of the process 100, wherein the machine learning techniques are configured to maintain an N-dimensional rule / action space and adjust the N-dimensional space based on the scores of the candidate compounds.

[0142] Process 400 may include the following steps: In step 402, a set of rules and / or operations for modifying a compound is received, which can be mapped into an N-dimensional rule / operation space; the process also receives data representing a first compound or a starting compound or a compound fragment, from which a set of one or more candidate compounds can be generated. In step 404, the set of rules / operations is encoded or mapped into the N-dimensional space. In step 406, the starting compound or fragment and / or one or more candidate compounds (if this is another pass of process 400 and / or process 100) are encoded and / or mapped into the N-dimensional space. For example, machine learning techniques can implement a neural network, in which the hidden layers in the neural network define the N-dimensional space, and the set of rules / operations and / or compounds is encoded or mapped into the N-dimensional space.

[0143] In step 408, a subset of rules / operations mapped in the N-dimensional operation space can be selected, which, when mapped in the N-dimensional rule / operation space, is the nearest neighbor of the compound. For example, k nearest neighbor rules / operations for the compound in the N-dimensional space can be selected. Assuming that more than one compound can be mapped in the N-dimensional rule / operation space, each mapped compound in the N-dimensional space can have a subset of k nearest neighbor rules / operations that have been mapped in the N-dimensional space. Therefore, one or more subsets of k nearest neighbor rules / operations corresponding to one or more compounds mapped in the N-dimensional space can be selected. That is, based on a distance function or a similarity function, k rules / operations that are considered to be closest to the compound mapped in the N-dimensional space are selected, where k≥1.

[0144] In step 410, each rule / operation subset in N-dimensional space can be applied to corresponding compound to generate one or more candidate compound sets. For example, the subset of rules / operations mapped in N-dimensional space can be decoded into the corresponding rule / operation of rule set, which can then be applied to the corresponding compound, or used to modify the corresponding compound accordingly. In step 412, process 400 can determine whether to perform one or more iterations of mapping, selecting and modifying steps 406, 408 and 410. If it is determined that more than one iteration is performed (e.g., "yes"), step 412 proceeds to step 406 for mapping / encoding each set of one or more candidate compounds into N-dimensional space. In step 412, if it is determined that further iterations of mapping are not performed, step 406 is selected and modified. Then, process 400 proceeds to step 414 for outputting data representing one or more candidate compound sets and / or corresponding rule / operation subsets, which can be used to modify the starting compound to generate one or more candidate compound sets.

[0145] The encoded machine learning technique may be implemented in step 104 of process 100 and configured to output a set of candidate compounds, which may be one or more candidate compounds, from step 104 of process 100 to a scoring step 106. Step 108 may determine whether to further repeat process steps 110, 104, and / or 106. Steps 108 or 110 may determine whether to update or further adjust the encoded machine learning technique to implement process 400 based on the scoring in step 110. Adjusting or updating the encoded machine learning technique based on the scoring further refines the N-dimensional vector space to better describe the location of rules / operations that are more suitable for generating / modifying compounds that more closely resemble or exhibit the desired properties.

[0146] Figures 4b-4e 4 is a schematic diagram illustrating example states 420, 430, and 450 of an N-dimensional operation / rule space 422 and how the encoder-based machine learning technique implementing process 400 can be used to generate a set of candidate compounds that are more likely to exhibit a desired property. As described above, N-dimensional vector space 422 is based on a set of rules / operations for modifying compounds, and N-dimensional vector space 422 can be created based on the structure of the encoder-based machine learning technique, wherein each of the different rules / operations in the set of rules / operations can be encoded or mapped to a unique N-dimensional vector in N-dimensional vector space 422.

[0147] As previously described, a rule for modifying a compound may include or represent data representing any principle, operation, rule, procedure, action, or any other command, code, instruction, or data format that can be used to describe modifying a compound from a first compound to a second compound. Thus, a set of rules for modifying a compound may include or represent data representing one or more rules for modifying a compound or a plurality of rules for modifying a compound.

[0148] For example, as mentioned above, multiple n>1 rules / operations for modifying compounds can form a rule set where R i is the i-th rule / operation for modifying the compound, which may include the following rules by way of example only and not limitation:

[0149] R1 - adding a first chemical element to the compound;

[0150] R2 - adding the first compound fragment to the compound;

[0151] R3 - removes the first chemical element from the compound;

[0152] R4—removing the first compound fragment from the compound;

[0153] R5 - adding a second chemical element to the compound;

[0154] R6 - adding a second compound fragment to the compound;

[0155] R7-removes a second chemical element from the compound;

[0156] R8 - removing a second compound fragment from the compound;

[0157] …

[0158] Ri -bonds added or reformed between atoms of a compound;

[0159] Rj-breaks or removes bonds between atoms of a compound;

[0160] …

[0161] R n-1 - any other rules or any other operations relating to the modification of a compound to form another compound; and

[0162] R n - Any other rules or any other operations related to modifying a compound to form a different compound.

[0163] This rule set Each rule / operation can be used one or more times to modify a compound from an initial compound to another compound. One or more rules (R i ), the rule set can define how to modify a first compound to form another compound based on the ordered sequence of rules.

[0164] refer to Figure 4b , shows an exemplary representation of a first state 420 of an N-dimensional vector space 422 in which a set of rules for modifying a compound may be encoded using machine learning techniques. The plurality of rules / operations in the rule / operation set are encoded or mapped to a unique N-dimensional vector in the N-dimensional vector space 422. The plurality of rules / operations in the rule / operation set are mapped to the N-dimensional vector space 42 in any suitable manner or using any one or more machine learning techniques. For example, a machine learning technique based on a neural network structure can be used to encode and / or map the plurality of rules / operations to the N-dimensional vector space 422 defined by the neural network structure, i.e., the so-called N-dimensional rule / operation space 422.

[0165] In step 404 of process 400, the coded machine learning technique applies the rule set Each rule / action in is mapped into the N-dimensional rule / action space 422. This is for example Figure 4b , where the rule / operation represented by R1 can be mapped to an N-dimensional vector operation / rule 424a, the rule / operation represented by R2 can be mapped to an N-dimensional vector operation / rule 424b, the rule / operation represented by R3 can be mapped to an N-dimensional vector 424c, etc., the rule / operation represented by Ri can be mapped to an N-dimensional vector operation / rule 424i, etc., and the rule / operation represented by R n-1 The rules / operations represented can be mapped to N-dimensional vector operations / rules 424m, and, by R n The represented rules / operations may be mapped to N-dimensional vector operations / rules 424n.

[0166] Figure 4c An exemplary second state 430 of the N-dimensional rule / action space 422 is shown, in which the encoded machine learning technique maps a compound (e.g., represented by the letter "C") into the N-dimensional rule / action space 422, e.g., step 406 of process 400. Compound C is represented as a compound prototype rule / action vector 432. Assuming that compound C has been mapped into the N-dimensional rule / action space 422 as a prototype rule / action vector 432, one or more rule / action vectors 424a-242n, i.e., prototype rule / action vectors 432, may now be selected that are likely to be near the location of compound C when mapped into the N-dimensional rule / action space 422.

[0167] In step 408 of process 400, once compound C is mapped to a compound prototype rule / action vector 432 in N-dimensional rule / action space 422, one or more rule / action vectors that are closest to compound prototype rule / action vector 432 can be selected. For example, a metric or distance metric / criterion, such as, but not limited to, the Euclidean distance between N-dimensional vectors, or any other metric or criterion, can be used to estimate the k nearest neighbor vectors of compound prototype rule / action vector 432. This metric or criterion is used to estimate the rule / action vectors that are closest to the compound prototype rule / action vector in the N-dimensional rule / action space. In this example, a distance metric 434 is determined between action / rule 424a and compound prototype rule / action vector 432, and a distance metric 434 between action / rule 424b and compound prototype rule / action vector 432 is determined as the minimum distance metric among the n distance metrics. In this example, for simplicity, k=2 is assumed, and therefore, two action / rule vectors are selected as the subset of action / rule vectors closest to compound prototype rule / action vector 432. Although k=2 in this example, this is merely exemplary and the present patent is not limited thereto, but those skilled in the art will appreciate that k≥1, and any value may be selected for k as long as it is greater than or equal to 1. In this example, operation / rule 424a and operation / rule 424b are selected as a subset of the rule / operation vector that may be used to modify compound C.

[0168] Figure 4d 442 and C2444, respectively.

[0169] Figure 4e4 is a schematic diagram of another exemplary state 450 in which compounds C1442 and C2444 are mapped into N-dimensional action / rule space 422. For example, in step 412 of process 400, it may be decided to perform multiple iterations of generating candidate compounds by mapping previously generated candidate compounds into N-dimensional space to find additional rules / actions to add to the subset of rules / actions. Encoded machine learning techniques map candidate compounds C1442 and C2444 into N-dimensional rule / action space 422, such as step 406 of process 400. Compounds C1442 and C2444 are represented as compound prototype rule / action vectors 452 and 454, respectively. Once mapped, further rules / actions may be selected by finding the k nearest neighbor action / rule vectors 424a-424n of the compound prototype rule / action vectors 452 and 454 representing compounds C1442 and C2444 in N-dimensional action / rule space 422.

[0170] In this example, a distance metric 456a is determined between operation / rule 424j and compound prototype rule / action vector 452, and a distance metric 456b is determined between operation / rule 424i and compound prototype rule / action vector 452 as the two smallest distance metrics (e.g., in this example, k=2) among n distance metrics to compound prototype rule / action vector 452. Operation / rule 424i and operation / rule 424j are selected to be included in a subset of rule / action vectors for further, individually, modifying C1 442. Operation / rule 424m is selected to be included in another subset of rule / action vectors for further, individually, modifying C2 444.

[0171] Figure 4e Also shown is an exemplary synthesis 460 (e.g., modification) of compound C1442 into compounds C3462 and C4464 using a selected subset of operation / rule vectors 424i and 424j (e.g., step 408 of method 400), and also shown is the synthesis of compound C2444 into compound C5466 using a selected subset of operation / rule vectors 424m. For example, compound C3462 would be the compound obtained by modifying compound C1442 with Ri (e.g., adding or reforming bonds between atoms of compound C1442 to form compound C3462). Compound C4464 would be the compound generated by modifying compound C1442 with Rj (e.g., breaking or removing bonds between atoms of compound C1442 to form compound C4464). Compound C5466 would be the compound generated by modifying compound C1442 with Rj (e.g., breaking or removing bonds between atoms of compound C1442 to form compound C4464). n-1 Compounds obtained by modifying compound C2444 (eg, breaking or removing bonds between atoms of compound C2444 to form compound C5465).

[0172] Process 400 can be concluded by outputting a set of candidate compounds and their corresponding subset of rule / operation sequences for modifying the compounds to form candidate compounds. For example, the output set of candidate compounds may include compounds C3462, C4464, and C5465. The rule sequence for generating compound C3462 from compound C includes (R1, Ri), the rule sequence for generating C4464 from compound C includes (R1, Rj), and the rule sequence for generating C5465 from compound C includes (R2, Rm), which can also be output from process 400.

[0173] As process 100 iterates, the encoding machine learning technology can be updated based on the scores of the output candidate compounds C3462, C4464, and C5465 (e.g., in steps 106 to 110). The scores indicate how close one or more candidate compounds C3462, C4464, and C5465 are to compounds that exhibit the desired properties and can be used to update the encoding machine learning technology and, thereby, improve the N-dimensional rule / action space. If the encoding machine learning technology generates a candidate compound that is closer to a compound that exhibits the desired properties, the encoding machine learning technology can be rewarded, or when the encoding machine learning technology generates a candidate compound that exhibits less than the desired properties, the encoding machine learning technology can be penalized. When the encoding machine learning technology is updated based on the scores of process 100, the encoding machine learning technology will improve or adjust the N-dimensional rule / action space, and thus the position of the rule / action vectors and / or prototype rule / action vectors that define the candidate compounds will change, so that the position of the prototype rule / action vector is closer to the rule / action vector that is more likely to result in the synthesis of a candidate compound that is closer to the desired properties.

[0174] Figure 5a The present invention is shown for use based on Figures 3a-4e Flowchart of an exemplary process 500 for generating one or more candidate compounds that exhibit one or more desired properties using a tree-encoding machine learning technique. The tree-encoding machine learning technique can be used in step 104 of process 100. The tree-encoding machine learning technique can be used by utilizing Figures 3a-3c The tree-based data structure shown in and Figures 4a-4e The N-dimensional rule / operation vector space shown in FIG is used to efficiently generate the candidate compound set used in step 106. The tree-based structure can be applied to efficiently generate, store and / or maintain a subset sequence of rules that can be used to generate each candidate compound, and the N-dimensional rule / operation vector space can be applied to efficiently select the best rule subset. The tree-encoding machine learning technology can be configured to map the set of rules / operations (e.g., through a neural network structure) and also map the compounds to the N-dimensional rule / operation vector space. As shown in FIG. Figures 4a-4eAs described, the N-dimensional rule / operation vector space can be updated or adjusted based on the scoring of candidate compounds according to the desired properties. Therefore, in each iteration of steps 104, 106, and 110 of process 100, the N-dimensional rule / operation vector space of the tree-encoding machine learning technique can be further improved, thereby increasing the probability that the selected subset of rules / operations can be used to generate candidate compounds that are closer to the desired properties.

[0175] The process 500 of the tree-encoding machine learning technique can be based on the following steps: In step 502, a first compound can be input and represented as the root node of a tree-like data structure, or a previously generated tree-like data structure representing a set of candidate compounds can be input into the tree-encoding machine learning technique, where each leaf node in the tree-encoding machine learning technique represents a candidate compound. In step 504, the encoding portion of the tree-encoding machine learning technique can use an N-dimensional rule / action space, where a set of rules / actions has been mapped to a set of N-dimensional rule / action vectors, to select a candidate compound to be compared with a reference compound. Figures 4a-4e A subset of rules / operations for modifying a compound is performed in a similar manner to that described in process 400. The first compound represented by the root node can be mapped into an N-dimensional rule / operation space as a compound prototype rule / operation vector via the encoding portion of the tree-encoding machine learning technique, and k nearest neighbor rule / operation vectors can be selected, where k ≥ 1. Thus, for the first compound, a subset of rules / operations can be generated by demapping the selected k nearest neighbor rule / operation vectors into corresponding rules / operations for modifying the first compound. In step 506, the selected subset of rules / operations can be used to generate one or more candidate compounds, which are represented as child nodes of the root node.

[0176] Alternatively or additionally, if a previously generated tree is input, then for each candidate compound or leaf node of the previously generated tree, the candidate compound represented by the leaf node can be mapped into the N-dimensional rule / action space as a compound prototype rule / action vector by the encoding portion of the tree-encoding machine learning technique, and k nearest neighbor rule / action vectors can be selected, where k≥1. Therefore, for each candidate compound, a subset of the selected rules / actions can be generated by demapping the selected k nearest neighbor rule / action vectors into corresponding rules / actions for modifying each candidate compound to generate other candidate compounds. In step 506, one or more subsets of the selected rules / actions can be used to generate one or more candidate compounds, which are represented as child nodes of each leaf node and become new leaf nodes of the tree-based structure.

[0177] In step 506, one or more child nodes of the current root node and / or leaf node may be generated based on one or more subsets of the selected rules / operations. n In step 508, a determination is made as to whether to continue parsing the tree-like data structure of the tree-encoding machine learning technique and move down to the next level in the tree and generate additional child nodes from the current leaf node (the current set of child nodes in the lowest level generated so far), or to output the current set of leaf nodes (or a selected set of child nodes) as the selected set of candidate compounds. If it is determined that the tree-like data structure is parsed (e.g., "yes"), the process 500 moves to step 510, in which the next level of the tree-like data structure is generated by repeating steps 504 and 508, instead of using process 400 for the current set of candidate compounds, the current set of candidate compounds is represented by the newly generated child nodes (or leaf nodes).

[0178] If, in step 508, it is determined to output a set of candidate compounds (e.g., "No"), then in step 512, the tree-encoding machine learning technique outputs a set of candidate compounds based on the selected set of child nodes or based on the current set of leaf nodes. The tree-encoding machine learning technique may also output a corresponding rule / operation sequence that can be generated by following a path from the root node of the tree (e.g., the first compound) to each of the selected child / leaf nodes.

[0179] As reference Figures 4a-4e As described, the encoding portion of the machine learning technique can be updated or adjusted based on the candidate compound's score for whether it exhibits or is closer to exhibiting the desired property. This adjustment / update will further improve the N-dimensional rule / operation vector space and make it more likely that additional candidate compounds will be generated that are closer to exhibiting the desired property. In each iteration of steps 104, 106, and 110 of process 100, the N-dimensional rule / operation vector space of the tree-encoding machine learning technique can be further improved, thereby increasing the likelihood that a selected subset of rules / operations can be used to generate candidate compounds that are closer to the desired property.

[0180] Figure 5b Is based on reference Figures 3a-5a Schematic diagram of an exemplary generation of a candidate compound set using the described tree-encoded machine learning technique. To generate the set of candidate compounds, where R iis the i-th rule / operation for modifying the compound. Each compound or candidate compound can be represented by a plurality of nodes 522, 524a-524n, 526a-526n and 528a-528n in the tree data structure 520, where each non-leaf node can have a plurality of rule edges R1, ..., R2 extending therefrom. n One or more of each rule edge. Each rule edge represents a rule from the rule set rules / operations and connect parent nodes to child nodes. Each parent node represents a compound, and each regular edge from each parent node to a child node represents a rule from the set of rules / operations. The rule / action of Can be executed on the compound represented by the parent node and generate the compound represented by the child node.

[0181] In this example, the root node of the tree 520 is created based on the first compound C. Initially, (e.g., see step 502 of process 500), the first compound C can be input into the tree-encoding machine learning technique, which represents the first compound C as the root node 522 of the tree data structure 520. The tree-encoding machine learning technique can then select the root node 522 from the rule / operation set. (For example, regular edges R1,…,R n ) selects a subset of rules / operations for use with reference to process 400 Figures 4a-4e Compound C is modified in a similar manner as described above. For example, tree-encoding machine learning techniques can be used to Encoded into an N-dimensional rule / action space 422, where the rule / action set is mapped to N-dimensional rule / action vectors 424a-424n. The first compound C represented by the root node 522 is also encoded into the N-dimensional rule / action space 422, where the first compound C is mapped to the compound prototype rule / action vector 432 in the N-dimensional rule / action space. Then, the tree-encoding machine learning technique can select a subset of rule / action vectors that are closest to the compound prototype rule / action vector 432. For example, k nearest neighbor rule / action vectors can be selected, where k ≥ 1. In this example, k = 2, and the k nearest neighbor rule / action vectors closest to the compound prototype rule / action vector 432 using (by way of example only and not limitation) distance metrics 434 and 436 are rule / action vectors 424a and 424b, respectively. Therefore, for the first compound C, a subset of rules / actions can be generated by demapping the selected k nearest neighbor rule / action vectors 424a and 424b to corresponding rules / actions R1 and R2 (or rule edges R1 and R2) that can be used to modify the first compound C.

[0182] The first compound C can be modified by generating the next level of the tree 520 based only on a selected subset of rules / operations (e.g., step 506 of process 500), which in this example includes rules / operations R1 and R2. The levels of the tree 520 can be created by extending only the rule edges corresponding to the selected subset of rules (e.g., rule edges R1 and R2) from the root node 522 to the corresponding child nodes 524a and 524b. The remaining rule edges are not extended. Therefore, no child nodes are created at this level of tree 520. In this example, child nodes 524a and 524b represent sets of candidate compounds. Thus, tree-encoding machine learning techniques have been able to effectively extract the candidate compound from the rule set. One or more rule / operation subsets are selected to generate one or more candidate compounds represented by child nodes.

[0183] The tree-encoding machine learning technique can continue to expand tree 520 to generate additional candidate compounds based on the current set of candidate compounds represented by child nodes 524a and 524b (e.g., see step 508 of process 500). In this example, the tree-encoding machine learning technique expands tree 520 to the next level to generate additional candidate compounds / child nodes. Thus, for each of child nodes 524a and 524b, the tree-encoding machine learning technique repeatedly maps the compounds and repeatedly selects a corresponding subset of k-nearest neighbor-based rules / operations.

[0184] For example, the first candidate compound represented by child node 524a is encoded into N-dimensional rule / action space 422, where first candidate compound 524a is mapped to first candidate compound rule / action vector 452 in N-dimensional rule / action space 422. Tree-encoding machine learning techniques can then select a subset of rule / action vectors from the set of rule / action vectors 424a-424n that are closest to the first candidate compound prototype rule / action vector 452. For example, k nearest neighbor rule / action vectors can be selected, where k ≥ 1. In this example, k = 2, and the k nearest neighbor rule / action vectors closest to the compound prototype rule / action vector 452 using (by way of example only and not limitation) distance metrics 456a and 456b are rule / action vectors 424i and 424j, respectively. Therefore, for first candidate compound 524a, the selected rule / action subset can be generated by demapping the selected k nearest neighbor rule / action vectors 424i and 424j into corresponding rules Ri and Rj.

[0185] The first candidate compound 524a can be modified by generating the next level of the tree 520 based only on a selected subset of rules / operations (e.g., step 506 of process 500). In this example, the selected subset of rules / operations includes rules / operations Ri and Rj. The next level of the tree 520 can be created by extending only the rule edges corresponding to the selected subset of rules (e.g., rule edges Ri and Rj) from child node 520a to corresponding new child nodes 526i and 526j. The remaining rule edges are not extended, so no child nodes are created at this level for this portion of the tree 520. In this example, child nodes 526i and 526j represent sets of another candidate compound C3 and C4, respectively.

[0186] Similarly, the second candidate compound represented by child node 524b is encoded into the N-dimensional rule / action space 422, where the second candidate compound 524b is mapped to the second candidate compound prototype rule / action vector 454 in the N-dimensional rule / action space 422. The tree-encoding machine learning technique can then select a subset of rule / action vectors from the set of rule / action vectors 424a-424n that are closest to the first candidate compound prototype rule / action vector 452. For example, k nearest neighbor rule / action vectors can be selected, where k≥1. In this example, k=2, and the k nearest neighbor rule / action vectors closest to the compound prototype rule / action vector 454 using (for example only and not limited to) the distance metric 458 are rule / action vectors 424m, respectively. Therefore, for the second candidate compound 524b, the selected k nearest neighbor rule / action vectors 424m can be demapped into the corresponding rule R n-1 (or regular edge R n-1 ) to generate the selected subset of rules / operations.

[0187] The second candidate compound 524b may be modified by generating the next level of the tree 520 based only on a selected subset of rules / operations (e.g., step 506 of process 500), which in this example includes rule / operation R n-1 This can be achieved by only including the rule edges corresponding to the selected rule subset (e.g., rule edges R n-1 ) extends from child node 520b to corresponding new child node 528m to create the next level of tree 520. The remaining regular edges are not expanded, so no more child nodes at this level are created for this portion of tree 520. In this example, child node 528m represents another set of candidate compounds C5.

[0188] Again, the tree-encoding machine learning technique may continue to expand the tree 520 to generate additional candidate compounds based on the current set of candidate compounds represented by child nodes 526i, 526j, and 526m at the current level (e.g., see step 508 of process 500). The tree-encoding machine learning technique may further expand the tree 520 to a predetermined number of m levels, where m>=1, by repeating the above process steps 504, 506, and 508 for each newly generated node at the previous level (e.g., performing m iterations of steps 504 to 508 based on the newly generated nodes at level m-1, where level 0 corresponds to the root node).

[0189] If the tree-encoding machine learning technique determines (e.g., in step 508 of process 500) that a set of candidate compounds for evaluation and scoring is to be output, for example, by step 106 of process 100, then the nodes of the mth level or current level of the tree 520 can be used as the set of candidate compounds. This determination can be based on the generation of a new level of the tree 520 for a predetermined number of iterations being performed, or based on a specific number of candidate compounds that have already been generated at a particular level of the tree 520. In any case, the set of candidate compounds can be output based on the leaf nodes of the current set of tree 520. For example, nodes 526i, 526j, and 528m are the most recently generated nodes of the tree 520, so these nodes 526i, 526j, and 528m can be used to output a set of candidate compounds corresponding to C3, C4, and C5. The tree-encoding machine learning technique can also output a corresponding rule / operation sequence, which can be generated by following the path from the root node of the tree (e.g., the first compound) to each of the selected child / leaf nodes. For example, each candidate compound has a corresponding rule / operation sequence that can be applied to the first compound C to generate each candidate compound. These may be generated by parsing or following paths through the tree 520 along regular edges connecting the root node representing the first compound C with corresponding child nodes representing candidate compounds.

[0190] For example, the set of candidate compounds that can be output based on tree 520 can include compounds corresponding to current leaf nodes 526i, 526j, and 528m, namely compounds C3, C4, and C5. The rule sequence for generating compound C3 from compound C includes (R1, Ri), which is a rule edge connecting node 522 (e.g., compound C) with node 526i (e.g., compound C3). The rule sequence for generating C4 from compound C includes (R1, Rj), which is a rule edge connecting node 522 (e.g., compound C) with node 526j (e.g., compound C4). The rule sequence for generating C5 from compound C includes (R2, Rm), which is a rule edge connecting node 522 (e.g., compound C) with node 528m (e.g., compound C5).

[0191] As process 100 iterates, the tree-encoding machine learning technique can be updated based on the scores of the output candidate compounds C3, C4, and C5 (e.g., in steps 106 to 110). The scores indicate how close one or more candidate compounds C3, C4, and C5 are to compounds exhibiting the desired property and can be used to update the tree-encoding machine learning technique, thereby improving the corresponding N-dimensional rule / action space 422. The tree-encoding machine learning technique can be rewarded if it generates a candidate compound that is closer to a compound exhibiting the desired property, or it can be penalized when it generates a candidate compound that does not exhibit the desired property as well. When the tree-encoding machine learning technique is updated based on the scores of process 100, the tree-encoding machine learning technique will improve or adjust the N-dimensional rule / action space 422, and thus the positions of the rule / action vectors and / or compound prototype rule / action vectors that define the candidate compounds will change, such that the prototype rule / action vectors are positioned closer to the rule / action vectors that are more likely to result in the synthesis of a candidate compound that is closer to or exhibits the desired property.

[0192] Once the tree-encoding machine learning technique has been updated, and / or process 100 determines that further iterations of steps 104 through 108 may be necessary, the tree-encoding machine learning technique may regenerate tree 520 based on the original starting compound C and the updated N-dimensional rule / action space 422. This is because the k-nearest neighbor search may select a different set of rules, assuming that the corresponding rule / action vectors and / or compound prototype rule / action vectors will have changed relative to each other. Alternatively or additionally, if the previous set of candidate compounds more closely resembles a compound exhibiting the desired properties, the tree-encoding machine learning technique may retain the previous tree-based structure 520 and simply further expand tree 520. For example, an additional m levels of tree 520 may be generated, or an additional predetermined number of new child nodes may be generated. In any case, given the update to the tree-encoding machine learning technique, the tree-encoding machine learning technique will make different decisions / selections regarding the rule / action edges and, therefore, generate a different set of child nodes, which in turn generate a further improved set of candidate compounds. The processes 100 and 500 can be further iterated with the current or rebuilt tree structure, the starting compound, or even with one or more promising candidate compounds, etc.

[0193] Figure 6a6 is a schematic diagram of a computing system 600 including a computing device or apparatus 602 according to the present invention. Computing device or apparatus 602 may include a processor unit 604, a memory unit 606, and a communication interface 608. Processor unit 604 is connected to memory unit 606 and communication interface 608. Processor unit 604 and memory 606 may be configured to implement one or more steps of one or more of processes 100, 300, 400, and / or 500 as described herein. Processor unit 604 may include one or more processors, controllers, or any suitable type of hardware for implementing computer-executable instructions to control apparatus 602 according to the present invention. Computing device 602 may be connected to a network 612 for communicating and / or operating with other computing devices / systems (not shown) for implementing the present invention.

[0194] Figure 6b is a schematic diagram illustrating an exemplary system 620 that can be used to implement one or more aspects of the design and generation of compounds, and / or to implement the methods described in reference Figures 1a-6a One or more of the methods, devices, and / or systems described herein. System 620 for designing compounds exhibiting one or more desired properties includes a compound generation module or device 622, a compound scoring module or device 624, a determination module or device 626, and an updated machine learning module or device 628 that can be connected together. Although these modules / devices are described separately, this is only for example, and those skilled in the art will understand that these modules can be combined or even further divided into other modules / devices according to application needs.

[0195] In operation, the compound generation module 622 is configured to use a machine learning technique that modifies a first compound based on a desired property and a set of rules for modifying the compound to generate a second compound. The compound scoring module 624 is configured to score the second compound based on the desired property (e.g., color). The decision module 626 is configured to determine whether to repeat the generation step based on the score. The update machine learning module 628 is configured to update the machine learning technique based on the score before repeating the generation step.

[0196] System 620 may further be configured to implement the methods described herein or as referenced herein. Figures 1a to 6a The system 620 can be configured such that the compound generation module 622, the compound scoring module 624, the determination module 626, and the update machine learning module 628 are further configured to implement the methods, processes, apparatuses, and / or systems described herein or in reference. Figures 1a-6aFor example, the compound generation module or device 622 may be further configured to implement functions, methods, processes and / or devices associated with generating candidate compounds using machine learning techniques (e.g., reinforcement learning techniques, tree-based reinforcement learning techniques, action space-based techniques, combinations thereof, etc.), as well as modifications thereof and / or as described herein or as referenced herein. Figures 1a-6a The compound scoring module or device 624 may be further configured to implement methods associated with scoring candidate compounds, etc. and / or as described herein or as referenced herein. Figures 1a-6a The decision module or means 626 may be further configured to implement functions, methods, processes and / or apparatus as described herein, associated with evaluating candidate compounds based on the scores, associated with determining which candidate compounds to proceed with, and associated with determining whether to proceed with generating additional candidate compounds, or associated with determining whether one or more candidate compounds have met the desired properties, and / or as described herein or as referenced herein. Figures 1a-6a The update machine learning module or means 628 may be further configured to implement functions, methods, processes and / or apparatus associated with updating machine learning techniques (which generate candidate compounds based on scores and / or calls associated with candidate compounds of the current iteration), and / or as described herein or as referenced herein. Figures 1a-6a The methods, processes and / or apparatus described herein are described.

[0197] Computing system 600 or system 620 may be a server system that may include a single server or a network of servers configured to implement the present invention as described herein. In some examples, the functionality of the server may be provided by a network of servers distributed over a geographical area (such as a globally distributed server network), and a user may be connected to an appropriate one of the server networks based on the user's location.

[0198] As reference Figures 1a to 6b As described in any one or more of the foregoing, and / or as described herein, the system 600 or 620 may include one or more further modifications, features, steps, and / or characteristics of the processes 100, 200, 300, 320, 340, 350, 400, 420, 430, 440, 450, 460, 500, 520 and / or the apparatus / systems 120, 210, 600, 620, computer-implemented methods thereof, and / or modifications thereof. For example, the compound generation module / device 622, the compound scoring module / device 624, the determination module / device 626, and / or the machine learning update module / device 628 may be configured to implement the foregoing. Figures 1a to 6band / or one or more further modifications, features, steps and / or features of any one or more of the processes 100, 200, 300, 320, 340, 350, 400, 420, 430, 440, 450, 460, 500, 520 and / or apparatus / systems 120, 210, 600, 620 as described herein, their computer-implemented methods, and / or modifications thereof.

[0199] Furthermore, hardware and / or software may be used to implement the Figures 1a to 6b and / or one or more further modifications, features, steps, and / or characteristics of any one or more of the processes 100, 200, 300, 320, 340, 350, 400, 420, 430, 440, 450, 460, 500, 520 and / or apparatus / systems 120, 210, 600, 620 as described herein, computer-implemented methods thereof, and / or modifications thereof. For example, reference may be made to hardware and / or software to implement the invention. Figures 1a-6b

[0014] The methods and / or processes described in one or more of the foregoing for designing compounds exhibiting one or more desired properties may be, for example (by way of example only and not limitation) as a computer-implemented method executed by one or more processors / processor units, or as an application requirement. Such apparatus, systems, processes and / or methods may be used to generate a machine learning model comprising data representing a method or process as described in conjunction with the computer-implemented methods, processes 100, 130, 500 and / or apparatus / systems 120, 300, 400, 600 and / or any of the methods / processes, steps of these processes, as described with reference to Figures 1a to 6b Any one or more of the foregoing and modifications thereof, and / or updates to the machine learning techniques as described herein, etc. Thus, a machine learning model may be obtained from the apparatus, system, and / or computer-implemented process, method as described herein.

[0200] For the sake of clarity, the above description discusses embodiments of the present invention with reference to a single user. It will be appreciated that in practice the system may be shared by multiple users, and potentially by a very large number of users simultaneously.

[0201] The above embodiments are fully automatic. In some examples, a user or operator of the system may manually indicate some steps of the method to be performed.

[0202] In the described embodiments of the invention, the system may be implemented as any form of computing and / or electronic device or apparatus. Such a device may include one or more processors, which may be microprocessors, controllers, or any other suitable type of processor for processing computer-executable instructions to control the operation of the device so as to collect and record routing information. In some examples, such as where a system-on-chip architecture is used, the processor may include one or more fixed function blocks (also known as accelerators) that implement a portion of the method in hardware (rather than software or firmware). Platform software including an operating system or any other suitable platform software may be provided at the computing-based device to enable execution of application software on the device.

[0203] The various functions described herein can be implemented using hardware, software, or any combination thereof. If implemented using software, the functions can be stored or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media may include, for example, computer-readable storage media. Computer-readable storage media may include volatile or non-volatile, removable or non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media can be any available storage medium that can be accessed by a computer. By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, flash memory, or other storage devices, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, or any other medium that can be used to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer. As used herein, disks include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs (BDs). In addition, propagation signals are not included within the scope of computer-readable storage media. Computer-readable media also include communication media, which include any media that facilitates the transfer of computer programs from one place to another. The connection may be, for example, a communication medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, it would be included in the definition of communications media. Combinations of the above should also be included within the scope of computer-readable media.

[0204] Alternatively or additionally, the functions described herein may be at least partially performed by one or more hardware logic components. For example, but not limitation, the hardware logic components that may be used may include a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), a system on a chip (SOC), a complex programmable logic device (CPLD), etc.

[0205] Although shown as a single system, it should be understood that the computing device can be a distributed system. Thus, for example, several devices can communicate via a network connection and can jointly perform the tasks described as being performed by the computing device.

[0206] Although the computing device is shown as a local device, it should be understood that the computing device may be accessed remotely via a network or other communication link (eg, using a communication interface).

[0207] As used herein, the term "computer" refers to any device or apparatus having processing capabilities such that it can execute instructions. Those skilled in the art will recognize that such processing capabilities are incorporated into many different devices, and thus the term "computer" includes any processing hardware / software, PCs, servers, mobile phones, personal digital assistants, and many other devices.

[0208] Those skilled in the art will recognize that the storage devices used to store program instructions can be distributed over a network. For example, a remote computer can store an example of a process described as software. A local or terminal computer can access the remote computer and download some or all of the software to run the program. Alternatively, the local computer can download multiple software programs as needed, or execute some software instructions at the local terminal and some at the remote computer (or computer network). Those skilled in the art will also recognize that all or part of the software instructions can be executed by dedicated circuits such as DSPs, programmable logic arrays, etc., using conventional techniques known to those skilled in the art.

[0209] It should be understood that the benefits and advantages described above may relate to one embodiment or may relate to several embodiments. The embodiments are not limited to embodiments that solve any or all problems or embodiments that have any or all benefits and advantages. Variations of the embodiments should be considered to be within the scope of the present invention.

[0210] Any reference to "an" item refers to one or more of those items. The term "comprising" is used herein to indicate that the method steps or elements identified are included, but these steps or elements are not an exhaustive list and the method or apparatus may include additional steps or elements.

[0211] As used herein, the terms "component" and "system" are intended to encompass a computer-readable data storage device configured with computer-executable instructions that, when executed by a processor, cause certain functions to be performed. Computer-executable instructions may include routines, functions, and the like. It should also be understood that a component or system may be located on a single device or distributed across multiple devices.

[0212] Furthermore, as used herein, the term "exemplary" is intended to mean "serving as an illustration or example of something."

[0213] Furthermore, to the extent the term "comprising" is used in either the detailed description or the claims, this term is intended to have a non-inclusive meaning in a manner similar to the term "comprising" when the term "including" is interpreted as a transitional word in the claims.

[0214] The accompanying drawings illustrate exemplary methods. Although these methods are shown and described as a series of operations performed in a particular order, it should be understood and appreciated that these methods are not limited by order. For example, some operations may occur in a different order than described herein. In addition, one operation may occur simultaneously with another operation. Furthermore, in some cases, not all operations may be required to implement the methods described herein.

[0215] Furthermore, the operations described herein may include computer-executable instructions that can be implemented by one or more processors and / or stored on a computer-readable medium. Computer-executable instructions may include routines, subroutines, programs, execution threads, etc. Furthermore, the results of the operations of these methods may be stored on a computer-readable medium, presented on a display device, etc.

[0216] The order of the steps of the methods described herein is exemplary, and the steps may be performed in any suitable order, or simultaneously where appropriate. In addition, steps may be added or substituted in any method, or individual steps may be deleted from any method, without departing from the scope of the subject matter herein. Aspects of any of the examples described above may be combined with aspects of any other example described to form additional examples without losing the effectiveness sought.

[0217] It should be understood that the above description of the preferred embodiments is provided by way of example only, and that various modifications may be made by those skilled in the art. The above description includes examples of one or more embodiments. Of course, it is not possible to describe every conceivable modification and variation of the above-described apparatus or method for the purposes of describing the above-described aspects, but those skilled in the art will recognize that many further modifications and permutations of various aspects are possible. Accordingly, the described aspects are intended to encompass all such variations, modifications, and variations that fall within the scope of the appended claims.

Claims

1. A computer-implemented method for designing compounds that exhibit one or more desired properties using machine learning techniques, characterized in that The method comprises: generating a second compound using the machine learning technique by modifying the first compound based on desired properties and a set of rules for modifying compounds, wherein the set of rules also includes data representing one or more operations associated with modifying the compound; scoring the second compound based on the desired property; determining whether to repeat the generating step based on the score; and updating the machine learning technique based on the scores before repeating the generating step, Wherein, generating the second compound further comprises: generating a tree data structure comprising a plurality of nodes and a plurality of edges, wherein each edge connects a parent node to a child node, wherein the parent node represents a compound, and each edge from the parent node to the child node represents an operation of a plurality of operations performed on the compound of the parent node that resulted in the compound of the child node, wherein a root node of the tree is the first compound and subsequent nodes correspond to the set of second compounds, expanding the tree data structure based on scoring one or more nodes corresponding to the second set of compounds, and performing a tree search on the tree data structure to generate the second set of compounds based on a set of one or more operations from the plurality of operations, And wherein generating one or more second compounds further comprises: Mapping the first compound and the set of operations to an N-dimensional operation space using the machine learning technique; selecting, by the machine learning technique, a subset of nearest neighbor operations of the first compound in the N-dimensional operation space when mapped in the N-dimensional operation space; and A subset of the operations in the N-dimensional space is applied to the first compound to generate a set of one or more second compounds.

2. The computer-implemented method of claim 1 , wherein: Determining whether to repeat the generating step is based on the score indicating that the second compound is closer to a compound exhibiting the desired property.

3. The computer-implemented method according to claim 1 or 2, wherein: Determining whether to repeat the generating step is based on the score indicating that the second compound exhibits the desired property.

4. The computer-implemented method according to claim 1 or 2, wherein: Determining whether to repeat the generating step further includes determining whether a predetermined number of iterations of repeating the generating step have been achieved.

5. The computer-implemented method according to claim 1 or 2, wherein: Determining whether to repeat the generating step further includes determining whether any further improvements to the second compound are possible based on the second compound exhibiting at least one or more of the desired properties.

6. The computer-implemented method according to claim 1 or 2, wherein: Generating a second compound further comprises generating a second set of compounds; and The second set of compounds is scored based on the desired property.

7. The computer-implemented method of claim 6, wherein: Also includes: The second compound set is ranked based on the scores, wherein generating second compounds further comprises generating other second compounds based on the second compound set with the highest rank.

8. The computer-implemented method of claim 1 or 2, wherein: The one or more operations include one or more operations from the following group: corresponding to an operation of adding a compound fragment or one or more atoms to the compound; operations corresponding to the removal of a fragment of a compound or one or more atoms of said compound; An operation corresponding to breaking or removing bonds between atoms of a compound; corresponds to operations that add or reform bonds between atoms of a compound; any other operation related to the modification of a compound to form another compound; as well as Any other operation related to modifying a compound to form a different compound.

9. The computer-implemented method of claim 1 or 2, wherein: The rule set and / or the one or more operations are selected to conform to desired structural, physical and / or chemical constraints that ensure that any modification of the compound and / or subsequently modified compound is feasible.

10. The computer-implemented method of claim 1 or 2, wherein: The set of rules and / or the one or more operations are based on a set of related chemical groups, the set of chemical groups comprising one or more of the following: one or more atoms; one or more molecules; one or more other compounds; one or more compound fragments; one or more keys; one or more functional groups; as well as One or more chemically related aspects of said compound.

11. The computer-implemented method of claim 1 , wherein: Generating the second set of compounds further includes selecting nodes associated with the selected set of operations for inclusion in the tree data structure.

12. The computer-implemented method of claim 1 or 2, wherein: The desired properties include one or more from the following group: The compound docks with another compound to form a stable complex; Specifically, the property is associated with a ligand that docks with a target protein, wherein the compound is the ligand; The compound docks or binds to one or more target proteins; The compound has a specific solubility or solubility range; as well as Any other property associated with a chemical compound that can be simulated using computer simulations based on the physical movement of atoms and molecules.

13. The computer-implemented method of claim 1 or 2, wherein: The score includes a certainty score, wherein when one or more of the second compounds substantially exhibit all of the one or more desired properties, one or more of the second compounds has a higher certainty score; when one or more of the second compounds substantially do not exhibit some of the one or more desired properties, one or more of the second compounds has a lower certainty score; when the second compound substantially exhibits some of the one or more desired properties, one or more of the second compounds has an uncertainty score between the higher certainty score and the lower certainty score.

14. The computer-implemented method of claim 13, wherein: The certainty score is a percentage certainty score, wherein the certainty score is a percentage certainty score, wherein the higher certainty score is 100%, the lower certainty score is 0%, and the uncertainty score is between the higher and lower certainty scores.

15. The computer-implemented method of claim 1 or 2, wherein: Generating the one or more second compounds further includes using a reinforcement learning technique to select one or more of a plurality of rules for modifying the first compound into the second compound.

16. The computer-implemented method of claim 1 or 2, wherein: At least a portion of the scoring is performed using one or more machine learning techniques.

17. The computer-implemented method of claim 1 or 2, wherein: The machine learning technology includes at least one machine learning technology or a combination of multiple machine learning technologies from the following group: a recurrent neural network configured to predict, starting from the first compound, the second compound that exhibits the desired set of properties; a convolutional neural network configured to predict, starting from the first compound, the second compound that exhibits the desired set of properties; a reinforcement learning algorithm configured to predict, starting from the first compound, the second compound exhibiting the desired set of properties; as well as Any neural network structure configured to predict, starting from said first compound, said second compound exhibiting said desired set of properties.

18. The computer-implemented method of claim 1 or 2, wherein: Scoring the second compound based on the one or more desired properties further comprises: analyzing the second compound for each of the desired properties; and An aggregate score for the second compound is calculated based on the analysis.

19. The computer-implemented method of claim 18, wherein: Analyzing the second compound further includes performing computer simulations related to the one or more desired properties of the second compound.

20. The computer-implemented method of claim 18, wherein: Analyzing the second compound further includes using a knowledge-based expert system to determine whether the second compound exhibits the one or more desired properties.

21. The computer-implemented method of claim 1 or 2, wherein: When generating a second compound using the machine learning technique, one or more first compounds are input to the machine learning technique.

22. The computer-implemented method of claim 1 or 2, wherein: Generating a second set of compounds using machine learning techniques also includes generating a second set of compounds using machine learning techniques based on the one or more desired properties and the set of rules.

23. A device comprising a processor, a memory unit and a communication interface, characterized in that: The processor is connected to the memory unit and the communication interface, wherein the processor and memory are configured to implement the computer-implemented method according to any one of claims 1 to 22.

24. A computer readable medium comprising data or instruction codes which, when executed on a processor, cause the processor to implement the computer-implemented method according to any one of claims 1 to 22.

25. A tangible computer-readable medium comprising data or instruction codes for using machine learning techniques to design chemical compounds exhibiting one or more desired properties, which, when executed on one or more processors, causes at least one of the one or more processors to perform at least one step of the following method: generating a second compound using the machine learning technique by modifying the first compound based on desired properties and a set of rules for modifying compounds, wherein the set of rules also includes data representing one or more operations associated with modifying the compound; scoring the second compound based on the desired property; determining whether to repeat the generating step based on the score; as well as updating the machine learning technique based on the scores before repeating the generating step, Wherein, generating the second compound further comprises: generating a tree data structure comprising a plurality of nodes and a plurality of edges, wherein each edge connects a parent node to a child node, wherein the parent node represents a compound, and each edge from the parent node to the child node represents an operation of a plurality of operations performed on the compound of the parent node that resulted in the compound of the child node, wherein a root node of the tree is the first compound and subsequent nodes correspond to the set of second compounds, expanding the tree data structure based on scoring one or more nodes corresponding to the second set of compounds, and performing a tree search on the tree data structure to generate the second set of compounds based on a set of one or more operations from the plurality of operations, And wherein generating one or more second compounds further comprises: Mapping the first compound and the set of operations to an N-dimensional operation space using the machine learning technique; selecting, by the machine learning technique, a subset of nearest neighbor operations of the first compound in the N-dimensional operation space when mapped in the N-dimensional operation space; and A subset of the operations in the N-dimensional space is applied to the first compound to generate a set of one or more second compounds.

26. The computer-readable medium of claim 25, wherein: It also comprises data or instruction codes which, when executed on a processor, cause the processor to implement one or more steps of the computer-implemented method of claims 2 to 22.

27. An apparatus for using machine learning techniques to design compounds that exhibit one or more desired properties, characterized in that The device comprises: a compound generation module configured to generate a second compound using the machine learning technique by modifying the first compound based on desired properties and a set of rules for modifying compounds, wherein the set of rules further includes data representing one or more operations associated with modifying the compound; a compound scoring module configured to score the second compound based on the desired property; a decision module configured to determine whether to repeat the generating step based on the score; and an update machine learning module configured to update the machine learning technique based on the score before repeating the generating step, Wherein, the compound generation module is further configured to: generating a tree data structure comprising a plurality of nodes and a plurality of edges, wherein each edge connects a parent node to a child node, wherein the parent node represents a compound, and each edge from the parent node to the child node represents an operation of a plurality of operations performed on the compound of the parent node that resulted in the compound of the child node, wherein a root node of the tree is the first compound and subsequent nodes correspond to the set of second compounds, expanding the tree data structure based on scoring one or more nodes corresponding to the second set of compounds, and performing a tree search on the tree data structure to generate the second set of compounds based on a set of one or more operations from the plurality of operations, And wherein generating one or more second compounds further comprises: Mapping the first compound and the set of operations to an N-dimensional operation space using the machine learning technique; selecting, by the machine learning technique, a subset of nearest neighbor operations of the first compound in the N-dimensional operation space when mapped in the N-dimensional operation space; and A subset of the operations in the N-dimensional space is applied to the first compound to generate a set of one or more second compounds.

28. The device according to claim 27, characterized in that The compound generation module, the compound scoring module, the decision module, and the update machine learning module are further configured to implement the computer-implemented method of claims 2 to 22.

Images