Machine learning system and method

Abstract

To directly generate samples (S, P) without any additional processing steps by learning of the underlying joint distribution p (S,P) of structure (S)-property (P) relationships, regarding a material design method using machine learning.SOLUTION: A generative adversarial network includes: a latent space vector generator configured to generate random vectors; a generator network configured to receive the random vectors as input and to generate a tuple including both a simulated structure and a simulated property; and a discriminator network configured to be trained from a database including real structures and real properties, to distinguish the simulated structure and the simulated property from the real structure and the real property, and to send a loss function to the generator network. The latent space vector generator, the generator network, and the discriminator network are arranged to learn an underlying joint probability distribution of structure-property p(S, P) simultaneously.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates generally to systems and methods for material design using machine learning. [Background technology]

[0002] Machine learning is being used to discover materials that meet the demand for materials with novel and / or advanced properties.

[0003] 1, the material discovery and design process is shown. A material property (P) of interest may be selected in advance in operation 101. For example, this may be a specific material's adhesion, resistivity, thermal conductivity, melting point, electromigration resistance, oxidation resistance, breakdown voltage, or a combination of material properties relevant to a particular application for the target material.

[0004] A list of candidate substances (M) is first collected in operation 102 based on prior knowledge and / or chemical intuition based on similarity to known substances with relevant values ​​of property P. For each substance M in the candidate list, various matching potential substance structures (S), such as crystal structures, orientations, etc., are generated in operation 103, and property P can be calculated for each potential structure at different lengths and time scales in operation 104. For example, (1) continuum-level simulations can be utilized when property P is calculated at a human-relevant time scale and large scales, and (2) atomistic simulations can be utilized when property P is calculated at the level of atoms and femtosecond time scales.

[0005] A decision is made in operation 105. If material M satisfies the target physical properties, it can be considered a favorable candidate for further experimentation and validation in operation 106. There may be more than one favorable candidate for property P with various tradeoffs, so the process may not stop here. If material M does not satisfy the target property P, the search continues until all materials in the candidate set have been simulated.

[0006] For purposes of this discussion, the process is shown in Figure 1. Figure 1 illustrates a "forward" mode of material design and discovery, which may be referred to as forward design. The list of candidate materials can be selected based on prior knowledge of materials that match the property P, or based on chemical intuition based on the similarity of the target material to other materials known to have suitable values ​​of P. While known materials / algorithms that map structure S to property P are used in forward design, such a selection process may still be tailored to the specific application, but may (1) lie outside the intuition of a domain expert or (2) be blind to a large number of target materials that differ from known materials with a given property.

[0007] Additionally, forward design may be limited by the time required to simulate a property P of a structure S. Time constraints in the possible physical world may mean that only a small portion can be simulated.

[0008] In this regard, there is a need to improve the process of material discovery using machine learning.

[0009] The background information is intended solely to enhance understanding of the inventive concepts described herein and should not be construed as an admission of the existence or relevance of prior art. Summary of the Invention [Problem to be solved by the invention]

[0010] One or more embodiments of the present invention relate to a machine learning system for materials design, in which a fundamental joint (or simultaneous) distribution p(S,P) of structure (S)-property (P) relationships is learned and utilized to immediately generate samples (S,P) in a single operation using generative techniques without additional processing steps (operations).

[0011] One or more embodiments of the present invention relate to a method for material design using machine learning, in which a fundamental joint distribution p(S,P) of structure (S)-property (P) relationships is learned and utilized to directly generate samples (S,P) in a single step using generative techniques without additional processing steps.

[0012] According to embodiments of the present invention, a subspace of structures (e.g., meaning the entire space representative of all possible structures) that meets or exceeds a target for a property P is identified by utilizing conditional generation (e.g., p(P)) or by randomly generating a large number of samples (S, P) and filtering (e.g., selecting) those that meet the target property criteria.

[0013] The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned above will be clearly understood by those skilled in the art from the following description. [Means for solving the problem]

[0014] According to an embodiment of the present invention, a machine learning system for generating a structure (S) associated with a target characteristic (P) includes a generative adversarial network (GAN), the GAN including: a latent space vector generator configured to generate a random vector; a generator network configured to receive the random vector as an input and generate tuples including all of the simulated structures and the simulated characteristics; and a discriminator network trained from a database including actual structures and actual characteristics, configured to distinguish the simulated structures and the simulated characteristics from the actual structures and the actual characteristics, and to transmit a loss function to the generator network, wherein the latent space vector generator, the generator network, and the discriminator network are configured to simultaneously learn an underlying joint probability distribution of the structure-characteristic p(S,P).

[0015] In one embodiment, the target property (P) comprises one or more physical quantities and the structure (S) comprises a representation selected from an encoded vectorized representation in which elements of a structure vector indicate atomic species at a particular location, a floating point vectorized representation in which successive tuples of a structure vector indicate variables (x, y, z) of Cartesian coordinates of a fixed atomic species, and a mixed representation comprising successive species and / or coordinate tuples.

[0016] In one embodiment, a machine learning system for generating a structure (S) according to a target property (P) includes a single generator network.

[0017] In one embodiment, the generator networks include a first generator network that generates the simulated structure and a second generator network that generates the simulated characteristic, and the first generator network and the second generator network are all configured to receive the random vectors from the same latent space vector generator.

[0018] In one embodiment, the generator network includes a first generator network and a plurality of second generator networks that generate the simulated structure, each of the plurality of second generator networks generating one of a plurality of simulated characteristics, and the first generator network and the plurality of second generator networks are all configured to receive the random vector from the same latent space vector generator.

[0019] In one embodiment, the classifier network is configured to output a floating point or discrete count value.

[0020] In one embodiment, a machine learning system for generating a structure (S) associated with a target property (P) includes computing devices including graphics processing units (GPUs) and / or field programmable gate arrays (FPGAs).

[0021] According to one or more embodiments of the present invention, a method for generating a structure (S) associated with a target property (P) using machine learning includes training a classifier network on a database including actual structures and actual target properties, generating simulated structures and simulated properties via a generator network, training the generator network so that the classifier network cannot distinguish between the simulated structures and the actual structures from the database, simultaneously learning a basic joint probability distribution for structure-property p(S,P), and generating the structure associated with the target property using the basic joint probability distribution.

[0022] In one embodiment, generating the structure associated with the target property includes generating the structure (S) associated with the target property (P) using a latent space partition p(P).

[0023] In one embodiment, generating the structure according to the target property includes randomly generating a number of samples (S, P) and filtering the samples that satisfy the target property (P).

[0024] In one embodiment, the target property (P) comprises a physical quantity and the structure (S) comprises a representation selected from an encoded vectorized representation in which elements of a structure vector indicate atomic species at specific locations, a floating point vectorized representation in which successive tuples of a structure vector indicate variables (x, y, z) of Cartesian coordinates of fixed atomic species, and a mixed representation comprising successive species and / or coordinate tuples.

[0025] In one embodiment, the method for generating a structure (S) associated with a target characteristic (P) using machine learning further includes generating a random vector via a latent space vector generator, wherein generating the simulated structure and the simulated characteristic via the generator network uses the random vector as an input, and training the generator network includes transmitting a loss function from the discriminator network to the generator network.

[0026] In one embodiment, the generator network includes a first generator network that generates the simulated structure and a second generator network that generates the simulated characteristic, and the first generator network and the second generator network are all configured to receive the random vectors from the same latent vector space generator.

[0027] According to one or more embodiments of the present invention, a machine learning system for generating a structure (S) associated with a target characteristic (P) includes a variational autoencoder (VAE), the VAE including an encoder network, a latent space, and a decoder network, wherein the encoder network receives as input a database including actual structures and actual characteristics, compresses the input to generate encoded vectors, and generates a mean and standard deviation of the encoded vector distribution in the latent space; the decoder network is trained from the database including the actual structures and the actual characteristics, decompresses the encoded vectors to generate the simulated structures and the simulated characteristics, distinguishes between the simulated structures and the simulated characteristics and the actual structures and the actual characteristics, and transmits a loss function to the encoder network; and the encoder network, the latent space, and the decoder network simultaneously learn a basic joint probability distribution of the structure-characteristic p(S,P).

[0028] In one embodiment, the target property (P) comprises one or more physical quantities and the structure (S) comprises a representation selected from encoded vectorized representations in which elements of a structure vector indicate atomic species at specific locations, floating point vectorized representations in which successive tuples of a structure vector indicate variables (x, y, z) of Cartesian coordinates of fixed atomic species, and mixed representations comprising successive species and / or coordinate tuples.

[0029] In one embodiment, the system includes computing devices including GPUs and / or FPGAs.

[0030] According to one or more embodiments of the present invention, a method for generating a structure (S) associated with a target property (P) using machine learning includes: inputting a database containing actual structures and actual properties to an encoder network; compressing the input to generate encoded vectors, a mean, and a standard deviation of the distribution of the encoded vectors in a latent space; training a decoder network with the database containing the actual structures and the actual properties; decompressing the encoded vectors to generate simulated structures and properties by the decoder network; transmitting a loss function to the encoder network; and simultaneously learning a basic joint probability distribution of the structure-property p(S,P).

[0031] In one embodiment, the method for generating a structure (S) associated with a target property (P) using machine learning further comprises generating the structure associated with the target property using a latent space partition p(P) to generate the structure associated with the target property (P).

[0032] In one embodiment, a method for generating a structure (S) according to a target property (P) using machine learning further comprises randomly generating a number of samples (S, P) to generate the structure according to the target property, and filtering the samples that satisfy the target property (P).

[0033] In one embodiment, the target property (P) comprises a physical quantity and the structure (S) comprises a representation selected from an encoded vectorized representation in which elements of a structure vector indicate atomic species at a particular location, a floating point vectorized representation in which successive tuples of a structure vector indicate variables (x, y, z) of Cartesian coordinates of a fixed atomic species, and a mixed representation comprising successive species and / or coordinate tuples.

[0034] Specific details of other embodiments are included in the description of the invention and the drawings. [Brief explanation of the drawings]

[0035] [Figure 1] 1 is an illustration of a forward design process. [Figure 2] FIG. 1 is a schematic diagram of a system and training process for a GAN according to one embodiment of the present invention. [Figure 3] FIG. 1 is a schematic diagram of a sample preparation process according to one embodiment of the present invention. [Figure 4] FIG. 1 is a schematic diagram of a system and training process for a GAN according to one embodiment of the present invention. [Figure 5] FIG. 1 is a schematic diagram of the inverse design approach for crystal structure prediction. [Figure 6] FIG. 1 is a schematic diagram of the inverse design approach to metasurfaces. [Figure 7] FIG. 1 is a schematic diagram of a system and training process using a VAE architecture according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0036] Other features and advantages of embodiments of the present invention will become more apparent with reference to the following description of the invention when considered in conjunction with the accompanying drawings, in which like reference numerals are used throughout the drawings to refer to like features and components, and numerical values ​​are not necessarily drawn to scale.

[0037] Exemplary embodiments will be described in more detail below with reference to the accompanying drawings. Throughout the specification, the same reference numerals refer to the same elements. However, the present invention is not limited to the embodiments disclosed below, but may be embodied in various different forms. These embodiments are provided solely to ensure a complete disclosure of the present invention and to fully convey the scope of the invention to those skilled in the art (hereinafter, "ordinary artisans"). The present invention is defined solely by the scope of the claims. Therefore, steps, elements, and techniques that are not essential for those skilled in the art to fully understand the aspects and features of the present invention will not be described. Unless otherwise specified, the same reference numerals refer to the same elements throughout the accompanying drawings and description, and therefore, description thereof will not be repeated.

[0038] The terms used in this specification are for the purpose of describing the embodiments and are not intended to limit the present invention. In this specification, the singular form includes the plural form unless otherwise specified in the context. The words "comprises" and / or "comprising" used in this specification do not exclude the presence or addition of one or more other components in addition to the components mentioned.

[0039] When describing embodiments of the present invention, the use of "may" is intended to refer to "one or more embodiments of the present invention." As used herein, "use," "using," and "used" may be considered synonymous with "utilize," "utilizing," and "utilized," respectively. Additionally, "exemplary" is intended to refer to an example or illustration.

[0040] Unless otherwise defined, all terms (including technical and scientific terms) used in this specification are to be used in the sense that they can be commonly understood by a person having ordinary skill in the art to which the present invention belongs. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless they are clearly and specifically defined.

[0041] Electronic or electrical devices and / or other related devices or components according to embodiments of the invention taught herein may be implemented using any suitable hardware, firmware (e.g., application-specific integrated circuits), software, or a combination of software, firmware, and hardware. The various components of these devices may be processes or threads that run on one or more processors in one or more computing devices, execute computer program instructions, and interact with other system components to perform the various functions described herein. The computer program instructions are stored in memory, which may be implemented in the computing device using standard memory devices such as, for example, volatile memory (RAM). The computer program instructions may also be stored on other non-transitory computer-readable media, such as, for example, CD-ROMs, flash drives, etc. Furthermore, within the scope of exemplary embodiments of the invention, those of ordinary skill in the art should recognize that the functionality of various computing devices may be combined or integrated into a single computing device, or the functionality of a particular computing device may be distributed across one or more other computing devices.

[0042] In the following description, the term "structure" can refer to the atomic structure, the arrangement of atoms in a regular or irregular pattern in a material and / or the spatial composition of atomic species in a compound.

[0043] In the following description, the term "property" may refer to physical and / or electronic structural properties such as boiling point, melting point, specific heat capacity, density, resistivity, thermal conductivity, electron transport, electromigration resistance, electron level density, phonon level density, mean free path of electron-phonon scattering, etc., calculated based on the atomic arrangement defined in the structure.

[0044] In the following description, the term "simultaneously" is used interchangeably with concurrently, synchronous, and / or coincidental in relation to steps or operations in a production process.

[0045] Throughout this specification, the term "joint probability distribution p(S,P)" refers to a probability distribution that indicates a structure-property relationship, where the structure S and the target property P can be discovered simultaneously as two separate events. The phrase "the joint distribution p(S,P) of Structure(S)-Property(P) relationships is learned simultaneously" refers to a system or method that learns the relationships p(S,P) directly and simultaneously. That is, when "the joint distribution p(S,P) of Structure(S)-Property(P) relationships is learned simultaneously," the joint distribution p(S,P) is not obtained from the individual probabilities p(S), individual probabilities p(P), conditional probabilities p(S|P), conditional probabilities p(P|S), or other machine learning distributions.

[0046] Also, the expression "sample (S, P)" refers to a sample having structure S and property P.

[0047] According to embodiments of the present invention, methods and systems for material design using machine learning are provided, in which a fundamental joint probability distribution p(S,P) of structure (S)-property (P) relationships is simultaneously learned (i.e., explicitly learned simultaneously) and used to directly generate samples (S,P) in a single step (operation) using generative techniques without additional processing steps (operations). A subspace of structures that meet or exceed a target property P is identified using a conditional generation of the distribution (e.g., p(P)) or by randomly generating a large number of samples (S,P) and filtering (e.g., selecting) those that meet the target property criteria.

[0048] In a related embodiment, an alternative approach to forward design can be inverse material design, where instead of using intuition and prior knowledge to find a set of candidate materials suited to an application, target properties P can be mapped to a structure S using a number of inverse design approaches. For example, inverse design approaches can be used for crystal structure prediction, as shown in the schematic diagram of FIG. 5.

[0049] A statistical inference-based approach 510 learns structure-property relationships from large databases and uses this learning to predict the probability that a particular composition of a material will have a particular property, e.g., a particular crystal structure type. Here, the output is the crystal (material) composition and the input is the crystal structure. For a large group of binary and ternary materials, a mapping of composition to ground state crystal structure is extracted with a machine learning model. The model is then used to predict the crystal type of a composition with an unknown structure.

[0050] In the genetic algorithm-based approach 530, evolutionary and / or other global optimization techniques such as particle swarm optimization (PSO), genetic algorithms, etc. are utilized to find a global optimum for the error function.

[0051]

number

[0052] Another approach can be a generative model-based approach 550. In a forward design problem, the probability distribution of interest is the conditional probability p(property|structure) of a given property, conditioned on the structure. The category of problems that find the output distribution when the input is known is called a discriminant problem. In the inverse approach, the condition is swapped to p(structure|property). Such problems are generative problems. A generative approach can include using a GAN to identify a geometric structure that matches the target optical spectrum, as shown in Figure 6.

[0053] Referring to FIG. 6, metasurface design can be transformed into neural network-mediated inverse design by utilizing both simulation and inverse design in a trial-and-error approach to construct a structure-transformation relationship for generating an optical spectrum from the metasurface, or vice versa. Both processes can be substituted with a deep neural network (DNN). The network structure for AI-based optical design can include three networks: a generator 610, a simulator 630, and a critic or evaluator 650, as shown in FIG. 6. The generator can accept a spectrum T and noise Z and generate possible patterns. The simulator can be a pre-trained network that approximates the transmission spectrum T for a given pattern at its output, and the critic can evaluate the distribution distance between the geometric data and the generator's pattern. During generator training, the generated patterns can be changed based on feedback from the simulator and critic. During the training process, valid patterns may be documented and qualified as candidate structures, but additional steps (operations) are required to identify structures with given characteristics.

[0054] Referring to Figures 5 and 6, a given process can only yield one of the conditional probability distributions of the structure (p(structure | property)) or the property (p(property | structure)). None of these approaches can generate or predict new structures using generative techniques without simultaneously obtaining the basic joint distribution p(S,P) of the structure (S)-property (P) relationship and without additional processing steps. In fact, such methods require many steps before a given structure (S) associated with a target property (P) can be identified.

[0055] In contrast, approaches according to embodiments of the present invention do not require an additional learning framework to bias the distribution for desired / target properties, and can perform inverse design entirely in a generative framework. For example, reinforcement learning may not be required (i.e., may not be included) herein.

[0056] According to some exemplary embodiments, the joint probability distribution of the structure and properties of any material can be obtained in a single operation, instead of involving intermediate and subsequent processing steps. Additionally, according to some exemplary embodiments, (S,P), i.e., the structure and properties, can be generated in a single operation without intermediate or subsequent processing steps.

[0057] According to one or more embodiments of the present invention, a method for simultaneously generating structure and properties includes simultaneously learning basic statistical properties of the joint probability distribution of the structure-property p(S,P), where no additional steps (operations) or computations are required between structure generation and property generation. According to embodiments of the present invention, the joint distribution p(S,P) is not learned sequentially by learning the distributions of p(S), p(S|P), p(P), and / or p(P|S).

[0058] According to embodiments of the present invention, the basic joint probability distribution p(S,P) can be learned by a generative framework selected from, but not limited to, generative adversarial networks (GANs), any suitable variants thereof such as deep convolutional GAN ​​(DCGAN), Wasserstein GAN (WGAN), and Fisher GAN (FGAN), variants thereof such as VAE, attention network, and / or combinations thereof such as VAE-GAN.

[0059] 2 is a schematic diagram of a system and training process for a GAN according to one embodiment of the present invention. The system includes a latent space vector generator (V) 210, a generator network (G) 230, and a discriminator network (D) 250.

[0060] In machine learning, an epoch is one iteration that covers all training data. The training data is divided into arrays of various sizes. The initial stage of every training epoch teaches the classifier what "true" examples are by utilizing actual structure-actual feature pairs. Subsequent stages of each training epoch are used to train a generator network (G) to generate simulated structures and simulated features that are indistinguishable from the actual pairs. By learning and training the structure-feature pairs together (simultaneously), the system can simultaneously learn the joint distribution of p(S,P).

[0061] During the training process (stage), the classifier network (D) is trained on the actual structure (R real ) and the actual characteristics (T real ) to learn the mapping function between the actual structure (R real ) and the actual characteristics (T real ) are simultaneously trained. For example, real materials with target properties are used to train the classifier network (D). real ) and the actual characteristics (T real ) to train the classifier network (D), the structure-property relationships can be learned simultaneously, i.e., not sequentially before or after learning the structure separately.

[0062] In machine learning, a latent space refers to an abstract multidimensional space containing feature values ​​(i.e., vectors) that encode meaningful internal representations of externally observed events (or phenomena), such as the structure and properties of a substance. The goal of latent space is to provide a computer with an understanding (or representation) of such events (structures and properties) through quantitative spatial representation / modeling. Each attribute of an event is represented by one or more vectors in the latent space, and events can be represented in multiple dimensions in the latent space.

[0063] In a GAN according to one embodiment of the present invention, a latent space vector generator (V) generates purely random numbers of a desired dimension. For example, the latent space vector generator (V) can output purely random vectors of any appropriate dimension. For example, in 10 dimensions, all random numbers are derived from a Gaussian distribution, i.e., a standard normal distribution, with a mean and unit variance of 0. In this framework, the concept behind using a random number generator is that the meaning of such random vectors is designated as "a posterion." During the training process, the vector space defined by the latent space vector generator is constrained so that the subspace of vectors in which a certain form of order is generated corresponds to a desired subspace, e.g., a subspace in which the target property has a certain structure. In one embodiment of the present invention, successful completion of the training process involves finding a subspace corresponding to the distribution of materials and structure-property pairs with the desired properties.

[0064] The latent space vector generator (V) generates random vectors that are mapped to the characteristic space and the structure space, and the generator network (G) generates simulated characteristic T based on the random vectors provided by the latent space vector generator (V). sim and the simulated structure R sim (T sim ,R sim) tuples of simulated samples. Throughout this specification, "T" and "P" are used interchangeably to refer to properties, and "R" and "S" are used interchangeably to refer to material structures.

[0065] These structure-feature pairs are fed into a trained discriminator network (D). If the discriminator network (D) can distinguish between the actual structure-feature pairs it was trained on and the simulated structure-feature pairs, i.e., the generated samples (T sim ,R sim ) is recognized as false, the difference is used to construct a loss function that is passed to the generator network (G), and the parameters are modified so that the generator network (G) generates samples that are closer to the real ones. The larger the difference, the larger the loss function, and the goal of training is to increase the error rate of the discriminator network (D), i.e., to "fool" the discriminator network so that it cannot distinguish the samples generated by the generator network (G) from real structure-feature pairs.

[0066] The loss function may be of any appropriate type that corresponds to the quantity being learned. For example, when learning continuous variables, the loss function may be a measure such as root mean square error. When learning categorical or class-based (e.g., qualitative) variables, the loss may be cross-entropy, Kullback-Leibler divergence, or a similar measure. In one embodiment, a negative binomial loss function may be used to train the generator network (G) when the output data is disjunctive. However, the present invention is not limited thereto, and other appropriate loss functions may be used.

[0067] According to one embodiment of the present invention, the structure and / or characteristics include or may include ordinal data or count data. The classifier network (D) may output floating-point (e.g., continuous floating-point) or discrete count values ​​instead of binary (true / false, 0 / 1) values. A loss function may be calculated based on the output of the classifier network (D). For example, a loss function for count data may be defined using negative binomial loss or Poisson loss.

[0068] In the latter part of the training process, in one embodiment, the discriminator network (D) may have a probability of correctly classifying generated samples as fake (simulated) or not, for example, 50% or more. The generator network (G) and the discriminator network (D) each act as an adversary trying to outperform the other, so the overall process is called adversarial sample generation. The generator network (G) attempts to generate real samples, while the discriminator network (D) attempts to discover fake samples. The two networks are maintained in an adversarial process until training is complete, i.e., until the generator network (G) generates samples that are sufficiently similar to the base (T, R) distribution.

[0069] Training is complete when the GAN is trained to the point where the simulated structure-property pairs are statistically indistinguishable from the actual structure-property pairs. At this stage, a joint distribution of p(S,P) is learned, and new materials that satisfy the target properties can be obtained by sampling the corresponding subspace.

[0070] Once the training process is complete, (T, R) examples that satisfy the target property P can be generated by sampling the latent space via a generator network (G) as shown in Figure 3. Referring to Figure 3, a latent space vector generator (V) 210 generates random vectors for a generator network (G) 230, which converts the random vectors into (S, P), i.e., structures (S) that have the target property (P). A subspace of structures that meet or exceed the target for property P can be identified using latent space partitioning (p(P)), or can be filtered to select candidates with the desired property (P) by simultaneously generating multiple candidate structure-property pairs (S, P).

[0071] According to some exemplary embodiments, the systems and methods can be applied to class-type, ordinal-type, integer-type, and / or count-type properties in addition to continuous properties. For example, if a property P is the ballistic electron emission of a material, the output indicative of the property according to one embodiment can be integer / count data. If the property is density of states, the output indicative of the property can be continuous data. If the property is a class to which a material belongs, the output indicative of the property can be class-type data.

[0072] According to one embodiment of the present invention, the target property can be any combination of two or more desired physical quantities, such as resistivity, density of states, etc., while the structure can be a floating-point vectorized representation in which a successive tuple of structure vectors represents the variables (x, y, z) in Cartesian coordinates of fixed atomic species. Systems and methods according to some exemplary embodiments can be applied to design any desired material, such as 1D, 2D, 3D, molecular, etc.

[0073] According to one embodiment of the present invention, the structure and feature generative framework generates all of the structures and features using a single generative network (G), which may be vectors that are concatenated together and fed to a discriminator network (D).

[0074] According to another embodiment of the present invention, a single generator network is used to generate the structure, while separate generator networks are used for each different characteristic among the multiple target characteristics. For example, a first generator network is used to generate the structure, a second generator network is used to generate the first characteristic, and a third generator network is used to generate the second characteristic. The structure, first characteristic, and second characteristic are then concatenated together.

[0075] 4 is a schematic diagram of a system and training process for a GAN according to an embodiment of the present invention. Referring to FIG. 4, a latent space vector generator (V) 410 is connected to a discriminator network (D) 450 and supplies structure (R) and feature (T) vectors to two generator networks (G) 431 and 433, which are respectively connected to the discriminator network (D) 450. The loss of the discriminator network (D) is supplied to the two generator networks (G) during the training process.

[0076] The loss function may be mean squared error (MSE), cross entropy, or any other suitable loss function.

[0077] In MSE, the error is calculated by taking the average over the entire dataset of the squared difference between the distribution of the data generated by the GAN and the distribution of the actual data.

[0078] In the cross-entropy loss function, the following mathematical formula can be used:

[0079]

number

[0080] Although the GNN is illustrated in FIGS. 2-4, the present invention is not limited thereto. According to one embodiment of the present invention, the fundamental joint probability distribution p(S,P) may be learned using a VAE. In one embodiment, the VAE includes an encoder network, a latent space, a decoder network, and a loss function. The encoder network is a neural network that outputs a compressed representation z of data x. The decoder network is a neural network that learns to reconstruct data (x) given the compressed representation (z). Through training, the loss function is reduced and minimized, and the VAE learns the probability distribution of the data.

[0081] More specifically, an encoder network consists of convolutional layers, which compress the input and output a dense representation of the input, called an encoding. The process by which the encoder network compresses the data (the encoded space, also known as the latent space) is called data compression or dimensionality reduction. During this process, the input is encoded into a vector with a probability distribution over the latent space. Points in the latent space are sampled with this distribution. The decoder network uses a deconvolutional layer, the opposite of the convolutional layer, to decompress the sampled points.

[0082] During the training process, pairs of actual structure and actual characteristics, i.e., initial data, are fed to the encoder network. That is, all of the actual structure and actual characteristics are fed to the encoder network. The encoder network converts the input into a vector distribution over a latent space and learns the mean and standard deviation of the probability distribution. The decoder network then receives the encoder output and outputs a reconstructed version of the input. The difference between the reconstructed version of the input and the actual input is backpropagated as a loss optimization function, and the neural network weights of the encoder and decoder are adjusted in an iterative optimization process, so that at the end of the training process, the decoder network is either an acceptable approximation of the initial data or indistinguishable from it.

[0083] The loss function consists of a "reconstruction term" (at the final layer) and a "regularization" term, where the reconstruction term tends to maximize the encoding-decoding scheme, and the regularization term tends to normalize the configuration of the latent space by making the distribution returned by such an encoder closer to a standard normal distribution. The training process minimizes the loss function.

[0084] After training is complete, i.e., when the decoder network output is indistinguishable from the initial data, the VAE architecture can be utilized to obtain new structure-property pairs. The training process results in a latent space that contains encoded representations of structure-property pairs, which, when passed through an appropriate decoder, will produce the desired output. To obtain new structures related to the target property, the learned joint probability distribution, mean, and standard deviation are used to provide input to the latent space that is decoded by the decoder to generate new structure-property pairs with a structure suitable for providing the target property. Filters can be used to select additional material structures. In this process, the decoder acts similarly to the generator network in a GAN, and the encoder is not utilized.

[0085] Figure 7 is a schematic diagram of a VAE architecture. Referring to Figure 7, during the training process, inputs are converted into latent distributions in a latent space 730 (encoded vectors) by an encoder network 710. The encoded vectors from the latent distributions are sampled by a decoder network 750 and converted into decoded outputs through a decoding process. During the generation process, points 730 from the latent space are selected and decoded by the decoder network 750 to generate newly generated structure-feature pairs.

[0086] According to one embodiment of the present invention, the generation process may include uncertainty estimation, which measures how certain the model is about its predictions. Prediction uncertainty can be estimated using any suitable available technique, such as active learning, Bayesian learning, etc.

[0087] In one embodiment, the property of interest may be electron transport (T), and structure (R) may be represented by a class-type value indicating the atomic species at a particular position in the unit cell. This configuration is suitable for studying the joint structure-transport distribution for alloys, such as binary, ternary, or quaternary alloys. T and R may be interlinked vectors. In this embodiment, T and R are discrete. For example, T may use integer values, while R may use class values.

[0088] According to some example embodiments, the method can be applied to a joint probability distribution of structures over different properties (e.g., structure <-> first property, second property). For example, the first characteristic can be electronic transmission and the second characteristic can be density of electronic levels. According to some exemplary embodiments, the method can be made conditional on certain characteristics. For example, the method can be used to find p(S, P1|P2), i.e., the probability distribution of (S, P1) given P2.

[0089] Additionally, structures obtained from methods according to embodiments of the present invention are not particularly limited to molecular structures, but may further include atomic structures, arrangements of atoms as regular or irregular patterns of matter, and may include the spatial composition of atomic species in a material (e.g., a compound) or sample of material.

[0090] The system may be implemented using any suitable computing devices, such as GPUs, FPGAs, or devices with other suitable hardware.

[0091] Although the embodiments of the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to the above embodiments and may be manufactured in various different forms, and those skilled in the art will understand that the present invention may be embodied in other specific forms without changing the technical spirit or essential features of the present invention. Therefore, the above-described embodiments should be understood to be illustrative in all respects and not limiting.

Claims

1. A machine learning system for generating a structure (S) related to a target property (P) including a GAN, The GAN includes a latent space vector generator configured to generate a random vector, a generator network configured to receive the random vector as an input and generate tuples including all simulated structures and simulated properties, and a discriminator network trained from a database including real structures and real properties, configured to distinguish the simulated structures and simulated properties from the real structures and real properties, and transmit a loss function to the generator network; the latent space vector generator, the generator network, and the discriminator network are configured to simultaneously learn a basic joint probability distribution of structure-property p(S,P) by learning a mapping relationship between structure (S) and property (S); A machine learning system for generating a structure (S) associated with a target property (P), in which the basic joint probability distribution for a first structure (S1) having a first property (P1) and a second property (P2) is used to discover a probability distribution p(S, P1|P2) for the first structure (S1) having the first property (P1) under a given condition of the second property (P2).

2. The target characteristic (P) includes one or more physical quantities, 2. A machine learning system for generating a structure (S) according to a target property (P) as described in claim 1, wherein the structure (S) comprises a representation selected from an encoded vectorized representation in which elements of a structure vector indicate atomic species at specific positions, a floating-point vectorized representation in which successive tuples of a structure vector indicate variables (x, y, z) of Cartesian coordinates of fixed atomic species, and a mixed representation including atomic species and coordinate tuples.

3. A machine learning system for generating a structure (S) according to a target property (P) according to claim 1 or 2, wherein the machine learning system comprises a single generator network.

4. the generator networks include a first generator network that generates the simulated structure and a second generator network that generates the simulated property; A machine learning system for generating a structure (S) according to a target characteristic (P) described in any one of claims 1 to 3, wherein the first generator network and the second generator network are configured to receive the random vectors from the same latent space vector generator.

5. the generator network includes a first generator network and a plurality of second generator networks that generate the simulated structure; the plurality of second generator networks each generating one of a plurality of simulated characteristics; A machine learning system for generating a structure (S) according to a target characteristic (P) described in any one of claims 1 to 3, wherein the first generator network and all of the plurality of second generator networks are configured to receive the random vectors from the same latent space vector generator.

6. A machine learning system for generating a structure (S) according to a target characteristic (P) according to any one of claims 1 to 5, wherein the classifier network is configured to output floating point or discrete count values.

7. The machine learning system for generating a structure (S) according to a target property (P) according to claim 6, wherein the machine learning system includes computing devices including GPUs and / or FPGAs.

8. training a classifier network on a database containing real structures and real target features; generating a simulated structure and a simulated property via a generator network; simultaneously learning a basic joint probability distribution of structure-property p(S,P) by learning a mapping relationship between structure (S) and property (S) when training the generator network so that the discriminator network cannot distinguish between the simulated structure and the actual structure in the database; 1. A method comprising: generating the structure according to the target characteristic using the fundamental joint probability distribution; A method for generating a structure (S) relating to a target property (P) using machine learning, in which the basic joint probability distribution for a first structure (S1) having a first property (P1) and a second property (P2) is used to discover a probability distribution p(S, P1|P2) for the first structure (S1) having the first property (P1) under a given condition of the second property (P2).

9. 9. The method for generating a structure (S) associated with a target property (P) using machine learning as described in claim 8, wherein generating the structure associated with the target property includes generating the structure (S) associated with the target property (P) using a latent space division p(P) corresponding to a material having the target property (P).

10. generating the structure according to the target characteristic includes: Randomly generate a number of samples (S, P), 10. A method for generating a structure (S) according to a target property (P) using machine learning according to claim 8 or 9, comprising filtering the samples that satisfy the target property (P).

11. The target property (P) comprises a physical quantity and the structure (S) comprises a representation selected from an encoded vectorized representation in which elements of a structure vector indicate atomic species at specific positions; The successive tuples of the structure vector contain floating-point vectorized representations of the Cartesian coordinate variables (x, y, z) of the fixed atomic species, or A method for generating a structure (S) according to a target property (P) using machine learning according to any one of claims 8 to 10, comprising a mixed representation comprising atom species and coordinate tuples.

12. further comprising generating a random vector via a latent space vector generator; generating the simulated structure and the simulated properties via the generator network using the random vector as an input; The method for generating a structure (S) according to a target property (P) using machine learning according to any one of claims 8 to 11, wherein training the generator network includes transmitting a loss function from the discriminator network to the generator network.

13. the generator networks include a first generator network that generates the simulated structure and a second generator network that generates the simulated property; 13. The method of generating a structure (S) according to a target property (P) using machine learning as described in claim 12, wherein the first generator network and the second generator network are all configured to receive the random vectors from the same latent vector space generator.

14. A machine learning system for generating a structure (S) relating to a target characteristic (P) including a VAE, The VAE includes an encoder network, a latent space, and a decoder network; the encoder network receives as input a database containing real structures and real properties, compresses the input to generate encoded vectors, and generates a mean and standard deviation of the distribution of the encoded vectors in the latent space; the decoder network is trained using the database containing the actual structures and the actual properties; decompressing the encoded vector to generate a simulated structure and a simulated property; distinguishing between the simulated structure and the simulated characteristics and the actual structure and the actual characteristics; transmitting the loss function to the encoder network; The encoder network, the latent space, and the decoder network simultaneously learn a basic joint probability distribution of structure-property p(S,P) by learning a mapping relationship between structure (S) and property (S); A machine learning system for generating a structure (S) associated with a target property (P), in which the basic joint probability distribution for a first structure (S1) having a first property (P1) and a second property (P2) is used to discover a probability distribution p(S, P1|P2) for the first structure (S1) having the first property (P1) under a given condition of the second property (P2).

15. The target property (P) comprises one or more physical quantities, and the structure (S) comprises a representation selected from an encoded vector representation in which elements of a structure vector indicate atomic species at specific positions; or The successive tuples of the structure vector contain floating-point vectorized representations of the Cartesian coordinate variables (x, y, z) of the fixed atomic species, or 15. A machine learning system for generating a structure (S) according to a target property (P) according to claim 14, comprising a mixed representation comprising atom species and coordinate tuples.

16. 16. A machine learning system for generating a structure (S) according to a target property (P) according to claim 14 or 15, wherein the machine learning system comprises computing devices including GPUs and / or FPGAs.

17. inputting a database containing real structures and real properties into an encoder network as input and compressing the input to generate encoded vectors, a mean, and a standard deviation of the distribution of the encoded vectors in a latent space; training a decoder network on said database containing said actual structures and said actual properties; decompressing the encoded vector to generate a simulated structure and simulated characteristics by the decoder network; transmitting a loss function to the encoder network; 1. A method comprising simultaneously learning a basic joint probability distribution of structure-property p(S,P) by learning a mapping relationship between structure (S) and property (S), A method for generating a structure (S) relating to a target property (P) using machine learning, in which the basic joint probability distribution for a first structure (S1) having a first property (P1) and a second property (P2) is used to discover a probability distribution p(S, P1|P2) for the first structure (S1) having the first property (P1) under a given condition of the second property (P2).

18. 18. The method for generating a structure (S) associated with a target property (P) using machine learning as described in claim 17, further comprising: generating the structure (S) associated with the target property (P) using a latent space division p(P) corresponding to a material having the target property (P).

19. 19. The method for generating a structure (S) according to a target property (P) using machine learning according to claim 17 or 18, further comprising randomly generating a number of samples (S, P) to generate the structure according to the target property, and filtering the samples that satisfy the target property (P).

20. The target characteristic (P) includes a physical quantity, 20. The method for generating a structure (S) according to a target property (P) using machine learning according to any one of claims 17 to 19, wherein the structure (S) comprises a representation selected from an encoded vectorized representation in which elements of a structure vector indicate atomic species at specific positions, a floating-point vectorized representation in which successive tuples of a structure vector indicate variables (x, y, z) of Cartesian coordinates of fixed atomic species, and a mixed representation including atomic species and coordinate tuples.

Images