Machine learning system and method for generating material structure of target material properties
By learning the structure-attribute joint distribution of materials, using GAN or VAE to generate the material structure of target attributes in a single step, solving the problems of time limitation and insufficient coverage in the prior art, and improving the efficiency and coverage of material design.
Patent Information
- Application Number
- CN202011064450.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-02-24
- Filing Date
- 2020-09-30
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2040-09-30
AI Technical Summary
Existing material design techniques have time constraints in meeting target physical properties, and the possibility of comprehensive exploration of the possible range of materials has led to many materials with potential advantages being ignored.
By learning the basic joint distribution of structure-attribute relationships, using a generative adversarial network (GAN) or a variational autoencoder (VAE) to directly generate material structures and attribute samples that meet the target attributes in a single step, avoiding additional processing steps. A direct mapping of material properties is realized.
The generation of material structures that meet the target attributes in a single step is achieved, which improves the efficiency and coverage of material design, and can identify structural subspaces that meet or exceed the target attributes, reducing dependence on expert intuition and time.
Smart Images

Figure CN112599208B_ABST
Abstract
Description
[0001] This application claims priority to and the benefit of U.S. Provisional Application No. 62 / 909,648, filed on October 2, 2019, entitled “GENERATIVE STRUCTURE-PROPERTY INVERSE COMPUTATIONAL CO-DESIGN OF MATERIALS,” the entire contents of which are incorporated herein by reference.
[0002] This application is related to U.S. patent application No. 16 / 798,245, filed on February 21, 2020, entitled “ELECTRONIC AND ATOMIC STRUCTURE COMPUTATION UTILIZING MACHINE LEARNING,” which claims priority to and the benefit of U.S. Provisional Application No. 62 / 902,324, filed on September 18, 2019, entitled “SEMICONDUCTOR DEVICE AND METHOD OF MAKING THE SAME,” and the entire contents of both the U.S. patent application and the U.S. Provisional Application are incorporated herein by reference. Technical Field
[0003] The present disclosure generally relates to systems and methods for materials design utilizing machine learning. Background Art
[0004] Machine learning has been used in materials discovery to meet the demand for materials with novel and / or improved properties.
[0005] Reference Figure 1 , shows a process for material discovery and design. A physical property of interest (P) may first be identified in act 101. For example, this may be adhesion of a given material to a target material, resistivity, thermal conductivity, melting point, electromigration resistance, oxidation resistance, breakdown voltage, or a combination of relevant physical properties of interest for a given application.
[0006] A list of candidate materials (M) may then be collected in act 102 based on prior knowledge and / or chemical intuition based on similarity to known materials having associated values for property P. For each material M in the candidate list, a variety of suitable potential material structures (S) (such as crystal structure, orientation, etc.) may be selected in act 103, and property P may be calculated for each of the potential structures in act 104 at different length and time scales, such as (1) using continuum level simulations (where property P is calculated at human-related time scales and large (micrometer or larger) length scales) or (2) using atomistic simulations (where property P is calculated at the atomic level and femtosecond time scales).
[0007] A decision is made in act 105. If material M meets the target physical property, material M may be considered a favorable candidate for further experimentation and validation in act 106. Because there may be more than one favorable candidate for property P with various tradeoffs, the process may not stop here. If material M does not meet the target for property P, the search continues until all materials in the candidate set have been simulated.
[0008] For the purpose of this description, Figure 1 The process shown in can be referred to as the "forward" mode of materials design and discovery (e.g., "forward design"). The list of candidate materials can be selected based on prior knowledge of materials suitable for property P or according to chemical intuition based on the similarity of the target material to other materials known to have suitable values of P. Although known physics / algorithms for mapping structure S to property P are used in forward design, such a selection process can be blind to many target materials that may still be suitable for a particular application but are either (1) outside the intuition of a domain expert, or (2) different from known materials with a given property.
[0009] Additionally, forward design may be limited by the time required to simulate a property P of a structure S. Due to time constraints, only a small fraction of the possible range of materials may be simulated.
[0010] Therefore, there is still a need to utilize machine learning to improve the process of materials discovery.
[0011] The above information in the Background section is only intended to enhance the reader's understanding of the inventive concepts described herein and therefore, it should not be construed as an admission of the existence or relevance of prior art. Summary of the invention
[0012] According to one aspect of one or more embodiments of the present disclosure, a machine learning system for material design is provided, wherein an underlying joint distribution p(S, P) of a structure (S)-property (P) relationship is learned and used to directly generate samples (S, P) in a single step (action) using a generative technique without any additional processing steps (actions).
[0013] According to one aspect of one or more embodiments of the present disclosure, a method for material design using machine learning is provided, wherein an underlying joint distribution p(S, P) of a structure (S)-property (P) relationship is learned and used to directly generate samples with (S, P) in a single step using a generative technique without the need for additional processing steps.
[0014] According to an embodiment of the present disclosure, a subspace of structures (e.g., from the "entire" space representing all possible structures) that meet or exceed a target property P is identified using conditional generation of a distribution (e.g., p(P)) or by randomly generating a large number of samples (S, P) and filtering (e.g., selecting) samples that meet the target property criteria.
[0015] According to one or more embodiments of the present disclosure, a machine learning system for generating a structure (S) of a target attribute (P) includes a generative adversarial network (GAN), wherein the GAN includes: a latent space vector generator configured to generate a random vector; a generator network configured to receive the random vector as an input and generate a tuple including both a simulated structure and a simulated attribute; and a discriminator network configured to be trained from a database including real structures and real attributes to distinguish the simulated structure and the simulated attributes from the real structure and the real attributes, and send a loss function to the generator network, wherein the latent space vector generator, the generator network, and the discriminator network are arranged to simultaneously learn the underlying joint probability distribution p(S,P) of the structure-attribute.
[0016] In one embodiment, the target property (P) includes one or more physical quantities, and the structure (S) includes a representation selected from an encoded vectorized representation, a floating-point vectorized representation, and a hybrid representation, wherein, in the encoded vectorized representation, elements of the structure vector represent atomic species at specific locations, in the floating-point vectorized representation, continuous tuples of the structure vector represent variables (x, y, z) of Cartesian coordinates of fixed atomic species, and the hybrid representation includes continuous species and / or coordinate tuples.
[0017] In one embodiment, the system includes a single generator network.
[0018] In one embodiment, the generator network includes a first generator network for generating simulated structures and a second generator network for generating simulated properties, and both the first generator network and the second generator network are configured to receive random vectors from the same latent space vector generator.
[0019] In one embodiment, the generator network includes a first generator network for generating a simulation structure and a plurality of second generator networks, each of the plurality of second generator networks is used to generate a simulation attribute among a plurality of simulation attributes, and the first generator network and the plurality of second generator networks are configured to receive random vectors from the same latent space vector generator.
[0020] In one embodiment, the discriminator network is configured to output a floating point or discrete count value.
[0021] In one embodiment, the system includes: a computing device including a graphics processing unit (GPU) and / or a field programmable gate array (FPGA).
[0022] According to one or more embodiments of the present disclosure, a method for generating a structure (S) of a target attribute (P) using machine learning includes: training a discriminator network using a database including real structures and real target attributes, generating simulated structures and simulated attributes through a generator network, training the generator network so that the discriminator network cannot distinguish the difference between the simulated structure and the real structure from the database, and simultaneously learning a basic joint probability distribution p(S,P) of the structure-attribute, and generating the structure of the target attribute using the basic joint probability distribution.
[0023] In one embodiment, the step of generating the structure of the target attribute includes: generating the structure (S) of the target attribute (P) using the latent space partition p(P).
[0024] In one embodiment, the step of generating a structure of target attributes includes: randomly generating a large number of samples (S, P) and filtering samples that satisfy the target attribute (P).
[0025] In one embodiment, the target property (P) includes a physical quantity and the structure (S) includes a representation selected from an encoded vectorized representation, a floating point vectorized representation, and a hybrid representation, wherein in the encoded vectorized representation, elements of the structure vector represent atomic species at specific positions, in the floating point vectorized representation, continuous tuples of the structure vector represent variables (x, y, z) of Cartesian coordinates of fixed atomic species, and the hybrid representation contains continuous (species, coordinate) tuples.
[0026] In one embodiment, the method further comprises: generating a random vector by a latent space vector generator, wherein the steps of generating simulated structures and generating simulated properties by a generator network utilize the random vector as input, and wherein the step of training the generator network comprises: sending a loss function from the discriminator network to the generator network.
[0027] In one embodiment, the generator network includes a first generator network for generating simulated structures and a second generator network for generating simulated properties, and both the first generator network and the second generator network are configured to receive random vectors from the same latent space vector generator.
[0028] According to one or more embodiments of the present disclosure, a machine learning system for generating a structure (S) of a target attribute (P) includes: a variational autoencoder (VAE), wherein the VAE includes an encoder network, a latent space, and a decoder network, wherein the encoder network is configured to: receive a database including real structures and real attributes as input, and compress the input to generate an encoding vector in the latent space, a mean value and a standard deviation of the distribution of the encoding vector, and the decoder network is configured to: be trained from a database including real structures and real attributes, decompress the encoding vector to create a simulated structure and simulated attributes, and distinguish the difference between the simulated structure and the simulated attributes and the real structure and the real attributes, and send a loss function to the encoder network, wherein the encoder network, the latent space, and the decoder network are arranged to simultaneously learn the underlying joint probability distribution p(S,P) of the structure-attribute.
[0029] In one embodiment, the target property (P) includes one or more physical quantities, and the structure (S) includes a representation selected from an encoded vectorized representation, a floating-point vectorized representation, and a hybrid representation, wherein, in the encoded vectorized representation, elements of the structure vector represent atomic species at specific locations, in the floating-point vectorized representation, continuous tuples of the structure vector represent variables (x, y, z) of Cartesian coordinates of fixed atomic species, and the hybrid representation includes continuous species and / or coordinate tuples.
[0030] In one embodiment, the system includes: a computing device including a graphics processing unit (GPU) and / or a field programmable gate array (FPGA).
[0031] According to one or more embodiments of the present disclosure, a method for generating a structure (S) of a target attribute (P) using machine learning includes: inputting a database including real structures and real attributes into an encoder network as input, and compressing the input to generate an encoding vector in a latent space, a mean value and a standard deviation of the distribution of the encoding vector, using the database including real structures and real attributes to train a decoder network, decompressing the encoding vector through the decoder network to create a simulated structure and simulated attributes, sending a loss function to the encoder network, and simultaneously learning the basic joint probability distribution p(S,P) of the structure-attribute.
[0032] In one embodiment, the method further comprises: generating a structure (S) of the target attribute (P) by using the potential space partition p(P) to generate the structure of the target attribute.
[0033] In one embodiment, the method further comprises: generating a structure of the target attribute by randomly generating a large number of samples (S, P) and filtering the samples satisfying the target attribute (P).
[0034] In one embodiment, the target property (P) comprises a physical quantity and the structure (S) comprises a representation selected from an encoded vectorized representation, a floating point vectorized representation, and a hybrid representation, wherein in the encoded vectorized representation, elements of the structure vector represent atomic species at specific positions, in the floating point vectorized representation, continuous tuples of the structure vector represent variables (x, y, z) of Cartesian coordinates of fixed atomic species, and the hybrid representation contains continuous (species, coordinate) tuples.
[0035] This summary is provided to introduce a selection of features and concepts of embodiments of the present disclosure that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. One or more of the described features may be combined with one or more other described features to provide a workable device. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] These and other features and advantages of the embodiments of the present disclosure will become more apparent by reference to the following detailed description when considered in conjunction with the following drawings. In the accompanying drawings, the same reference numerals are used throughout the drawings to refer to the same features and components. The drawings are not necessarily drawn to scale.
[0037] Figure 1 This is a diagram of the forward design process.
[0038] Figure 2 is a schematic diagram of a system and training process for a generative adversarial network (GAN) according to an embodiment of the present disclosure.
[0039] Figure 3 is a schematic diagram of a sample generation process according to an embodiment of the present disclosure.
[0040] Figure 4 is a schematic diagram of a system and training process for a generative adversarial network (GAN) according to an embodiment of the present disclosure.
[0041] Figure 5 is a schematic diagram of the inverse design method for crystal structure prediction.
[0042] Figure 6 It is a schematic representation of the inverse design method of a metasurface.
[0043] Figure 7 is a schematic diagram of a system and training process utilizing a VAE architecture according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0044] Hereinafter, example embodiments will be described in more detail with reference to the accompanying drawings, in which the same reference numerals always represent the same elements. However, the present invention may be implemented in various different forms and should not be construed as being limited to the embodiments shown herein. On the contrary, these embodiments are provided as examples so that this disclosure will be thorough and complete, and will fully convey the aspects and features of the present invention to those skilled in the art. Therefore, unnecessary processing, elements and techniques for a complete understanding of the aspects and features of the present invention by those of ordinary skill in the art may not be described. Unless otherwise stated, throughout the drawings and written descriptions, the same reference numerals represent the same elements, and therefore the descriptions of the same reference numerals may not be repeated.
[0045] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. It will also be understood that when the terms "comprise" and "comprising" are used in this specification, it indicates the presence of the stated features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their groups.
[0046] The use of "may" when describing embodiments of the present invention means "one or more embodiments of the present invention." As used herein, the terms "use," "using," and "used" may be considered synonymous with the terms "utilize," "utilizing," and "utilized," respectively. In addition, the term "exemplary" is intended to mean an example or illustration.
[0047] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. It will also be understood that, unless expressly defined as such herein, terms (such as those defined in general dictionaries) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and / or this specification, and should not be interpreted in an idealized or overly formal sense.
[0048] Any suitable hardware, firmware (e.g., application specific integrated circuits), software, or a combination of software, firmware, and hardware may be utilized to implement electronic or electrical devices and / or any other related devices or components according to the embodiments of the present invention described herein. The various components of these devices may be processes or threads, which run on one or more processors in one or more computing devices, execute computer program instructions, and interact with other system components for performing the various functions described herein. Computer program instructions are stored in a memory that can be implemented in a computing device utilizing a standard memory device (such as, for example, a random access memory (RAM)). Computer program instructions may also be stored in other non-temporary computer-readable media (such as, for example, a CD-ROM, a flash drive, etc.). In addition, those skilled in the art will recognize that, without departing from the spirit and scope of the exemplary embodiments of the present invention, the functions of various computing devices may be combined or integrated into a single computing device, or the functions of a particular computing device may be distributed on one or more other computing devices.
[0049] In the following description, the term "structure" may refer to atomic structure, arrangement of atoms in a material in a regular or irregular pattern, and / or spatial composition of atomic species in a material (eg, a compound).
[0050] In the following description, the term "property" may refer to physical properties (such as boiling point, melting point, specific heat capacity, density, resistivity, thermal conductivity, electron transport, high resistance to electron migration, electron energy level density, phonon energy level density, mean free path of electron-phonon scattering, etc.), and / or electronic structure properties calculated based on the atomic arrangement defined in the structure.
[0051] In the following description, the term "simultaneously" is used interchangeably with concurrently, synchronously, and / or coincidentally in time with respect to steps or actions in a generation process.
[0052] Throughout the present disclosure, the term "joint probability distribution p(S,P)" as used herein refers to a probability distribution representing a structure and attribute relationship, wherein, as two separate events, both the structure S and the target attribute P are to be established simultaneously. The statement "the joint distribution p(S,P) of the structure (S)-attribute (P) relationship is learned simultaneously" refers to a system or method that learns the relationship p(S,P) directly, simultaneously or concurrently. That is, when the "joint distribution p(S,P) of the structure (S)-attribute (P) relationship is learned simultaneously", the joint distribution p(S,P) is not obtained from the individual probabilities p(S), the individual probabilities p(P), the conditional probabilities p(S|P), the conditional probabilities p(P|S) or another machine-learned distribution.
[0053] Furthermore, the expression “sample (S, P)” refers to a sample having a structure S and a property P.
[0054] According to an embodiment of the present disclosure, a method and system for material design using machine learning are provided, wherein the underlying joint probability distribution p(S, P) of the structure (S)-attribute (P) relationship is simultaneously learned (i.e., explicitly simultaneously learned) and is used to directly generate samples (S, P) in a single step (action) using a generative technique without any additional processing steps (actions). The subspace of structures that meet or exceed the target of attribute P is then identified using conditional generation of the distribution (e.g., p(P)), or by randomly generating a large number of samples (S, P) and filtering (e.g., selecting) samples that meet the target attribute criteria.
[0055] In an example of the prior art, an alternative to forward design can be inverse material design. Here, instead of using intuition and prior knowledge to find a set of candidate materials suitable for an application, a target property P can be mapped to a structure S by using a number of inverse design methods. For example, inverse design methods can be used to Figure 5 A schematic diagram showing the predicted crystal structure.
[0056] Method 510 based on statistical inference requires learning structure-property relationships from a large database and using this learning to predict the probability that a specific composition of a material has a specific property (e.g., a specific crystal structure type). Here, the output is the crystal (material) composition, and the input is the crystal structure. From a large group of binary and ternary materials, the composition-to-ground state crystal structure mapping is extracted into a machine learning model. The model is then used to predict the crystal type of a composition with an unknown structure.
[0057] In the genetic algorithm based method 530, evolution and / or other global optimization techniques (such as particle swarm optimization (PSO), genetic algorithm, etc.) are used to find the global optimal value O(σ) of the error function:
[0058]
[0059] Here, the attribute α or the sum of multiple attributes is calculated in the space of candidate structures σ. The global minimum (ie, the structure that optimizes the difference between the desired attribute and the calculated attribute) is obtained by utilizing a heuristic optimization technique.
[0060] Another approach may be a generative model-based approach 550. In the forward design problem, the probability distribution of interest is p(attribute|structure), i.e., the conditional probability of the structure given the attribute. Such problems that seek the distribution of the output when the input is known are called identification problems. In the inverse approach, the condition is the opposite p(structure|attribute), and the opposite p(structure|attribute) is sought. Such problems are generative problems. Figure 6 As shown in , the generation method may include using a generative adversarial network (GAN) to identify geometric structures that match the target spectrum.
[0061] Reference Figure 6 By using both simulation and inverse design to set the structure-property relationship to generate spectra from the metasurface, metasurface design can be transformed from a trial-and-error approach to a neural network mediated inverse design, and vice versa. Both processes can be replaced by deep neural networks. Figure 6 As shown in , the architecture of a network for artificial intelligence (AI) based optical design may include three networks: a generator 610, a simulator 630, and a critic 650. The generator 610 may accept a spectrum T and noise z and generate a feasible pattern. The simulator 630 may be a pre-trained network that approximates the transmission spectrum for a given pattern at its output. And the critic 650 may evaluate the distance l of the distribution between the geometric data and the pattern from the generator. When training the generator 610, the generated pattern may change according to the feedback obtained from the simulator 630 and the critic 650. Valid patterns may be recorded during the training process and may be smoothed to qualify as candidate structures, but additional steps (actions) will be needed to identify structures for given properties.
[0062] exist Figure 5 to Figure 6 In the methods shown in , only one of the conditional probability distribution of structure p(structure|attribute) and the conditional probability distribution of attribute p(attribute|structure) is obtained in a given process. None of these methods utilize generative techniques to simultaneously obtain the underlying joint distribution p(S,P) of the structure (S)-attribute (P) relationship, and no additional processing steps are required, for example, in generating or predicting new structures. In fact, in these methods, multiple steps are required before a given structure (S) for a target attribute (P) is identified.
[0063] In contrast, the method according to the embodiment of the present disclosure can perform inverse design entirely in a generative framework without requiring any additional learning framework to bias the distribution toward the desired / target attribute. For example, reinforcement learning may not be required (i.e., may not be included) in the present application (e.g., the current method).
[0064] According to some example embodiments, the joint probability distribution of the structure and properties of any material can be obtained in a single step (action) without intermediate steps (actions) and post-processing. In addition, according to some example embodiments, (S, P) (i.e., structure and property) can be generated in a single step (action) without any intermediate or post-processing steps (actions).
[0065] According to one or more embodiments of the present disclosure, a method for simultaneously generating structure and attributes includes simultaneously learning the basic statistical properties of the joint probability distribution p(S, P) of the structure-attribute, wherein no additional steps (actions) or computational processing are required between the generation of the structure and the generation of the attribute. According to an embodiment of the present disclosure, the joint distribution p(S, P) is not learned sequentially by learning the distributions of p(S), p(S|P), p(P) and / or p(P|S).
[0066] According to an embodiment of the present disclosure, the basic joint probability distribution p(S,P) can be learned (or estimated) by a generation framework, and the generation framework is selected from but not limited to: a generative adversarial network (GAN), any suitable variant of a generative adversarial network (such as, deep convolutional GAN (DCGAN), Wasserstein GAN (WGAN), Fisher GAN (FGAN), etc.), a variational autoencoder (VAE), any suitable variant of a variational autoencoder (such as, an attention network) and / or a combination thereof (such as, VAE-GAN).
[0067] Figure 2 2 is a schematic diagram of a system and training process for generating an adversarial network (GAN) according to an embodiment of the present disclosure. The system includes a latent space vector generator (V) 210, a generator network (G) 230, and a discriminator network (D) 250.
[0068] In machine learning, an epoch is an iteration through all the training data. The training data is split into batches of various sizes. The initial pass in each training epoch teaches the discriminator what a "real" example looks like using real structure and real attribute pairs. Subsequent passes in each training epoch are used to train the generator network (G) to produce simulated structures and simulated attributes that are indistinguishable from the real structure and real attribute pairs. By learning and training together (simultaneously) using structure-attribute pairs, the system can simultaneously learn the joint distribution of p(S, P). That is, the latent space vector generator (V) 210, the generator network (G) 230, and the discriminator network (D) 250 can simultaneously learn the underlying joint probability distribution p(S, P) of structure-attribute.
[0069] In the training process, the real structure (R real ) and the real property (T real ) to train the discriminator network (D) to learn the mapping function between the two. For example, the discriminator network (D) is trained using real materials with target properties. real ) and the real property (T real ) are used to train the discriminator network (D), and the structure-attribute relationship can be learned simultaneously (e.g., not sequentially after learning the structure, or separately before or after learning the structure).
[0070] In machine learning, latent space refers to an abstract multidimensional space containing eigenvalues (i.e., vectors) that encodes meaningful internal representations of externally observed events (such as the structure and properties of materials). The latent space is intended to provide computers with an understanding (or representation) of these events (structure and properties) through quantitative spatial representation / modeling. Each attribute of an event is represented by one or more vectors in the latent space, and an event can be represented using multiple dimensions in the latent space.
[0071] When utilizing a generative adversarial network (GAN) according to an embodiment of the present disclosure, a latent space vector generator (V) generates pure random numbers of a desired dimension. For example, the latent space vector generator (V) may output pure random vectors of any suitable dimension (e.g., dimension 10) such that all random numbers are drawn from a Gaussian distribution (i.e., a standard normal distribution) with zero mean and unit variance. The idea behind utilizing a random number generator in this framework is that the meaning of these random vectors is assigned "a posteriori," i.e., after the discriminator network (D) rejects the sample because it is false (i.e., simulated). During the training process, some form of order is being imposed on the vector space defined by the latent space vector generator so that the subspace of the generated vectors corresponds to the desired subspace (e.g., a subspace of structures having target properties). In one embodiment of the present disclosure, successful completion of the training process finds a subspace corresponding to the distribution of structure-property pairs using materials having desired properties.
[0072] The latent space vector generator (V) generates random vectors mapped to the attribute space and the structure space, and the generator network (G) generates a network including simulated attributes T based on the random vectors provided from the latent space vector generator (V). sim and simulated structure R sim (T sim ,R sim ) tuples. Throughout the current description, "T" and "P" are used interchangeably to represent properties, and "R" and "S" are used interchangeably to represent material structures.
[0073] These structure / attribute pairs are then fed into a trained discriminator network (D). If the discriminator network (D) can distinguish the difference between the real structure-attribute pairs and the simulated structure-attribute pairs, that is, if the generated samples (T sim , R sim ) is identified as fake, the difference is used to construct the loss function passed to the generator network (G), forcing the generator network (G) to modify its parameters to produce more realistic samples. The larger the difference, the larger the loss function. The training goal is to increase the error rate of the discriminator network (D) (i.e., "fooling" the discriminator network (D) so that the discriminator network (D) cannot distinguish between samples generated by the generator network (G) and real structure-attribute pairs).
[0074] The loss function can be any suitable type corresponding to the quantity being learned. For example, if a continuous variable is being learned, the loss function can be a quantity such as the root mean square error. If a categorical (e.g., qualitative) variable is being learned, the loss can be a cross entropy, Kullback-Leibler divergence, or a similar measure. In one embodiment, when the output data is discrete, a negative binomial loss function can be utilized to train the generator network (G). However, embodiments of the present disclosure are not limited thereto, and other suitable loss functions can be utilized.
[0075] According to an embodiment of the present disclosure, the structure and / or attribute may include or consist of ordinal or count data. The discriminator network (D) may output floating point (e.g., continuous floating point) or discrete count values instead of binary (true / false, 0 / 1) values. A loss function may be calculated based on the output of the discriminator network (D). For example, a loss function for count data may be defined using a negative binomial loss or a Poisson loss.
[0076] At the end of the training process, in one embodiment, the discriminator network (D) may have, for example, a 50% or greater probability of correctly classifying the generated samples as fake (simulated) or not fake. Because the generator network (G) and the discriminator network (D) act as adversaries each trying to outperform the other, the entire process is referred to as adversarial generation of samples. The generator network (G) tries to create realistic samples, while the discriminator network (D) tries to find fake samples, and the two networks are locked in the adversarial process until training is complete (i.e., until the generator network (G) generates reasonably realistic samples of the underlying (T, R) distribution).
[0077] When the GAN is trained to the point where its simulated structure-property pairs are statistically indistinguishable from the real structure-property pairs, training is complete. At this stage, the joint distribution of p(S,P) has been learned, and new materials satisfying the target properties can be obtained by sampling the corresponding subspace.
[0078] Once the training process is complete, Figure 3 The generator network (G) shown in Figure 2 samples the latent space to create (T, R) samples that satisfy the target property P. Figure 3, the latent space vector generator (V) 210 generates a random vector for the generator network (G) 230, and the generator network (G) 230 transforms the random vector into (S, P) (i.e., a structure (S) having a target attribute (P)). The subspace of the structure that meets or exceeds the target of attribute P can be identified by the use of latent space partitioning (p(P)) or by the simultaneous (or concurrent) generation of a large number of candidate structure / attribute pairs (S, P), followed by filtering to select candidates with the desired attribute (P). Here, p(P) represents the probability of an attribute in the latent space that satisfies the target attribute P. The term "latent space partitioning" means dividing the latent space according to the attribute P so that the subspace that satisfies the target attribute P can be identified. In this method, the latent joint probability distribution p(S, P) of the structure (S)-attribute (P) relationship is first learned simultaneously through machine learning, and then the subspace of the structure that meets or exceeds the target of attribute P can be identified using conditional generation of the distribution p(P). Since the joint probability distribution p(S,P) of the structure (S)-attribute (P) relationship has been learned, the structure that can provide the target attribute P can be identified accordingly.
[0079] According to some exemplary embodiments, in addition to continuous attributes, the system and method may also be applied to categorical attributes, ordinal attributes, integer attributes, and / or count attributes. For example, when the attribute P is the ballistic electron transport of the material, the output representing the attribute according to the embodiment may be integer / count data. When the attribute is the density of states, the output representing the attribute may be continuous data. When the attribute is the class to which the material belongs, the output representing the attribute may be categorical data.
[0080] According to an embodiment of the present disclosure, the target property can be any combination of two or more desired physical quantities (such as resistivity, density of states, etc.), and the structure can be an encoded vectorized representation, wherein, in the encoded vectorized representation, the elements of the structure vector represent the atomic species at a specific position.
[0081] According to an embodiment of the present disclosure, the target property may be any combination of two or more desired physical quantities (such as resistivity, density of states, etc.), and the structure may be a floating point vectorized representation, wherein, in the floating point vectorized representation, a continuous tuple of structure vectors represents a variable (x, y, z) in Cartesian coordinates of a fixed atomic species. The systems and methods according to some example embodiments may be applicable to designing any desired material (such as 1D, 2D, 3D, molecules, etc.).
[0082] According to an embodiment of the present disclosure, the target attribute may be any combination of two or more desired physical quantities (such as resistivity, density of states, etc.), and the structure may be a hybrid representation containing continuous species and / or coordinate (e.g., (species, coordinate)) tuples. In one example, the hybrid representation includes continuous atomic species and / or Cartesian coordinate tuples.
[0083] According to an embodiment of the present disclosure, a generative framework for structure and attributes utilizes a single generative network (G) to generate both structure and attributes. Both structure and attributes can be vectors concatenated with each other and then fed to a discriminator network (D).
[0084] According to another embodiment of the present disclosure, a single generative network is used for structure generation, while different generative networks are used for each different attribute among multiple target attributes. For example, a first generative network is used for structure generation, a second generative network is used for first attribute generation, and a third generative network is used for second attribute generation. The structure, first attribute, and second attribute are then cascaded together.
[0085] Figure 4 is a schematic diagram of a system and training process for generating an adversarial network (GAN) according to an embodiment of the present disclosure. Figure 4 , the latent space vector generator (V) 410 is fed into two generator networks (G) 431 and 433, which separately create structure (R) vectors and attribute (T) vectors, respectively, which are concatenated and fed into the discriminator network (D) 450. During the training process, the loss from the discriminator network (D) 450 is fed into the two generator networks (G) 431 and 433. Although Figure 4 , but in a non-limiting example, the generator network may include a first generator network (G) 431 for generating a simulated structure and a plurality of second generator networks (G) 433, each of the plurality of second generator networks (G) 433 for generating a simulated attribute in a plurality of simulated attributes. The first generator network (G) 431 and the plurality of second generator networks (G) 433 are both configured to receive random vectors from the same latent space vector generator (V) 410.
[0086] The loss function may be mean squared error (MSE), cross entropy, or any other suitable loss function.
[0087] In MSE, the error is calculated by averaging over the entire dataset the square of the difference between the distribution of the data generated by the GAN and the distribution of the real data.
[0088] In the cross entropy loss function, the following mathematical formula can be used:
[0089]
[0090] In the cross entropy loss function, y i Indicates the actual value, Indicates the simulated value. When the actual label is 1 (y i =1), the second half of the function disappears, and when the actual label is 0 (y i =0), the first half is discarded. The cross entropy for binary or two-class prediction problems is calculated as the average cross entropy over all examples.
[0091] Although in Figures 2 to 4 A generative adversarial network (GAN) is shown in , but the present disclosure is not limited thereto. According to an embodiment of the present disclosure, a variational autoencoder (VAE) may be used to learn the underlying joint probability distribution p(S, P). In one embodiment, a variational autoencoder (VAE) includes an encoder network, a latent space, a decoder network, and a loss function. The encoder network is a neural network that outputs a compressed representation z of the data x. The decoder network is a neural network that learns to reconstruct the data x given the compressed representation z. Through training, the loss function is reduced or minimized, and the VAE learns the probability distribution of the data.
[0092] In more detail, the encoder network consists of convolutional layers that compress the input and outputs a dense representation of the input (called encoding). This process of the encoder network compressing the data (from the initial space to the encoded space (also called latent space)) is called data compression or dimensionality reduction. In this process, the input is encoded as a vector with a probability distribution over the latent space. Points from the latent space are sampled from this distribution. The decoder network uses deconvolutional layers and decompresses the sampled points, which are the inverse of the convolutional layers.
[0093] During the training process, the true structure and true attribute pairs (i.e., initial data) are fed to the encoder network. That is, both the true structure and the true attribute are fed to the encoder network. The encoder network transforms the input into a vector distribution on the latent space and learns the mean and standard deviation of the probability distribution. The decoder network then takes the output of the encoder and outputs a reconstructed version of the input. The difference between the reconstructed version of the input and the actual input is back-propagated as a loss function to adjust the weights of the neural networks of the encoder and decoder in an iterative optimization process so that at the end of the training process, the output of the decoder network is an acceptable approximation of the initial data or cannot be distinguished from the initial data.
[0094] The loss function consists of a "reconstruction term" (on the final layers) that tends to make the encoding-decoding scheme as efficient as possible and a "regularization term" (on the latent layers) that tends to regularize the organization of the latent space by making the distribution returned by the encoder close to a standard normal distribution. Over the training process, the loss function is minimized.
[0095] After training is complete (i.e., the output of the decoder network cannot be distinguished from the initial data), the VAE architecture can be used to obtain new structure-attribute pairs. Through the training process, the latent space becomes an encoded representation (of the structure-attribute pairs) that produces the desired output when passed through a suitable decoder. To obtain a new structure for the target attribute, the learned joint distribution probability, mean, and standard deviation are used to provide input from the latent space that will be decoded by the decoder to generate (partake) new structure-attribute pairs with a structure suitable for providing the target attribute. Filters can be used to further select the material structure. In this process, the decoder will act similarly to the generator network of the generative adversarial network, and the encoder is not used.
[0096] Figure 7 This is a schematic diagram of the VAE architecture. Figure 7 During the training process, the input is transformed by the encoder network 710 into a latent distribution (as an encoded vector) in the latent space 730. The encoded vector from the latent distribution is sampled by the decoder network 750 and transformed into a decoded output through the decoding process. During the generation process, points from the latent space 730 (as sampled vectors) are selected and decoded by the decoder network 750 to produce newly generated structure / attribute pairs.
[0097] According to an embodiment of the present disclosure, the generation process may include uncertainty estimation, which gives a measure of how confident the model is in its predictions. The uncertainty of the predictions may be estimated using any suitable available technique, such as active learning, Bayesian learning, etc.
[0098] In one embodiment, the property of interest may be electron transport (T), while the structure (R) may be represented by a categorical value representing the atomic species at a particular position in the unit cell. Such a configuration is suitable for studying the connected structure-transport distribution of alloys (such as binary, ternary, or quaternary alloys). Both T and R may be vectors concatenated with each other. In this embodiment, T and R are discrete. For example, T may take an integer value, while R may take a categorical value.
[0099] According to some exemplary embodiments, the method may be applied to the joint probability distribution of a structure and a plurality of attributes (e.g., structure ←→ (attribute 1, attribute 2)), a relationship. For example, the first attribute may be electron transport, and the second attribute may be the density of electron energy levels. According to some exemplary embodiments, the method may be conditioned on a particular attribute. For example, the method may be used to find p(S, P1|P2) (i.e., the probability distribution of (S, P1) given P2), etc.
[0100] In addition, the structure obtained from the method according to the embodiments of the present disclosure is not specifically limited to molecular structure, and may also include atomic structure, atomic arrangement in regular or irregular patterns in a material, and spatial composition of atomic species in a material (e.g., a compound) or a sample of a material.
[0101] The system may be implemented using any suitable computing device, such as a computing device with a graphics processing unit (GPU), a field programmable gate array (FPGA), or other suitable hardware.
[0102] Although the present invention has been described in detail with specific reference to exemplary embodiments of the present invention, the exemplary embodiments described herein are not intended to be exhaustive or to limit the scope of the present invention to the exact form disclosed. Those skilled in the art and technical field to which the present invention belongs will understand that changes and modifications in the described structures and methods of assembly and operation may be implemented without departing from the principles, spirit and scope of the present invention as set forth in the claims and their equivalents.
Claims
1. A machine learning system for generating a material structure of a target material property, the machine learning system comprising a generative adversarial network, in, Generative adversarial networks include: A latent space vector generator, configured to generate random vectors, a generator network configured to receive as input the random vector and generate a tuple including both a simulated material structure and a simulated material property, and a discriminator network configured to be trained from a database including real material structures and real material properties to distinguish the simulated material structures and the simulated material properties from the real material structures and the real material properties, and to send a loss function to the generator network, Among them, the latent space vector generator, the generator network and the discriminator network are arranged to simultaneously learn the underlying joint probability distribution of material structure-material properties. Wherein, the generator network is further configured to generate a material structure of a target material property by utilizing the basic joint probability distribution.
2. The machine learning system according to claim 1, in, The target material properties include one or more physical quantities, and the material structure includes a representation selected from an encoded vectorized representation, a floating-point vectorized representation, and a hybrid representation, wherein, in the encoded vectorized representation, elements of the material structure vector represent atomic species at specific positions, in the floating-point vectorized representation, continuous tuples of the material structure vector represent variables of Cartesian coordinates of fixed atomic species, and the hybrid representation includes continuous tuples of atomic species and / or Cartesian coordinates.
3. A machine learning system according to claim 1 or claim 2, in, The generator network includes a single generator network.
4. A machine learning system according to claim 1 or claim 2, in, The generator network includes a first generator network for generating simulated material structures and a second generator network for generating simulated material properties, and Both the first generator network and the second generator network are configured to receive random vectors from the same latent space vector generator.
5. The machine learning system according to claim 1 or claim 2, in, The generator network includes a first generator network for generating a simulated material structure and a plurality of second generator networks, each of the plurality of second generator networks for generating one simulated material property of a plurality of simulated material properties, and The first generator network and the plurality of second generator networks are both configured to receive random vectors from the same latent space vector generator.
6. The machine learning system according to claim 1 or claim 2, in, The discriminator network is configured to output either floating point or discrete count values.
7. The machine learning system according to claim 1 or claim 2, in, The machine learning system is implemented using a computing device including a graphics processor and / or a field programmable gate array.
8. A method for generating a material structure of a target material property using machine learning, the method include: Generate a random vector through the latent space vector generator, The generator network receives random vectors as input and generates simulated material structures and simulated material properties. training a discriminator network using a database including real material structures and real target material properties; The generator network is trained so that the discriminator network cannot distinguish the difference between the simulated material structure and the simulated material properties and the real material structure and the real material properties from the database, Simultaneously learning the underlying joint probability distribution of material structure-material properties through a latent space vector generator, a generator network, and a discriminator network; and The base joint probability distribution is used to generate a material structure of a target material property.
9. The method according to claim 8, in, The step of generating a material structure of a target material property includes generating a material structure of the target material property by utilizing a latent space partition.
10. The method according to claim 8, in, The step of generating a material structure with a target material property includes randomly generating a plurality of samples having a material structure and material properties and selecting a sample that satisfies the target material property.
11. The method according to any one of claims 8 to 10, in, The target material properties include physical quantities, and the material structure includes a representation selected from an encoded vectorized representation, a floating point vectorized representation, and a hybrid representation, wherein in the encoded vectorized representation, elements of the material structure vector represent atomic species at specific positions, in the floating point vectorized representation, continuous tuples of the material structure vector represent variables of Cartesian coordinates of fixed atomic species, and the hybrid representation contains continuous tuples of atomic species and / or Cartesian coordinates.
12. The method according to any one of claims 8 to 10, in, The steps to train the generator network include sending the loss function from the discriminator network to the generator network.
13. The method according to claim 12, in, The generator network includes a first generator network for generating simulated material structures and a second generator network for generating simulated material properties, and Both the first generator network and the second generator network are configured to receive random vectors from the same latent space vector generator.
14. A machine learning system for generating a material structure for a target material property, the machine learning system include: Variational Autoencoders, Among them, the variational autoencoder includes an encoder network, a latent space, and a decoder network. The encoder network is configured to receive a database including real material structures and real material properties as input, and compress the input to generate an encoding vector in a latent space, a mean value and a standard deviation of a distribution of the encoding vectors, The decoder network is configured to: be trained from a database including real material structures and real material properties, decompress the encoded vector to create a simulated material structure and simulated material properties, and distinguish the difference between the simulated material structure and the simulated material properties and the real material structure and the real material properties, and send a loss function to the encoder network, The encoder network, latent space, and decoder network are arranged to simultaneously learn the underlying joint probability distribution of material structure-material properties. Wherein, the target material properties include one or more physical quantities, and the material structure includes a representation selected from an encoded vectorized representation, a floating-point vectorized representation, and a hybrid representation, wherein, in the encoded vectorized representation, the elements of the material structure vector represent atomic species at specific positions, in the floating-point vectorized representation, continuous tuples of the material structure vector represent variables of Cartesian coordinates of fixed atomic species, and the hybrid representation includes continuous atomic species and / or Cartesian coordinate tuples.
15. The machine learning system according to claim 14, in, The machine learning system is implemented using a computing device including a graphics processor and / or a field programmable gate array.
16. A method for generating a material structure of a target material property using machine learning, the method include: A database including real material structures and real material properties is input to the encoder network as input, and the input is compressed to generate an encoding vector in the latent space, the mean and standard deviation of the distribution of the encoding vector, The decoder network is trained using a database that includes real material structures and real material properties. The encoded vector is decompressed through a decoder network to create simulated material structures and simulated material properties, Send the loss function to the encoder network, and At the same time, learn the basic joint probability distribution of material structure and material properties, Wherein, the target material properties include physical quantities, and the material structure includes a representation selected from an encoded vectorized representation, a floating-point vectorized representation, and a hybrid representation, wherein, in the encoded vectorized representation, elements of the material structure vector represent atomic species at specific positions, in the floating-point vectorized representation, continuous tuples of the material structure vector represent variables of Cartesian coordinates of fixed atomic species, and the hybrid representation contains continuous atomic species and / or Cartesian coordinate tuples.
17. The method according to claim 16, in, The step of generating a material structure of a target material property includes generating a material structure of a target material property by using a latent space partition.
18. The method according to claim 16, in, The step of generating a material structure with a target material property includes randomly generating a plurality of samples having a material structure and material properties and selecting a sample that satisfies the target material property.
