A method, system, and device for predicting material properties based on graph neural networks.

Abstract

This invention discloses a method, system, and device for predicting material properties based on graph neural networks, belonging to the field of material prediction and analysis. The method includes: acquiring the crystal structure of the material; encoding, initializing, and standardizing the crystal structure to obtain a graph network of corresponding atoms; constructing a structural feature prediction model, determining the network hyperparameters of the model, and setting the network evaluation algorithm, activation function, and optimizer; training and optimizing the model based on the hyperparameters, evaluation algorithm, activation function, and optimizer; inputting the material data to be predicted into the trained and optimized model, and outputting the predicted material properties. This invention can achieve accurate characterization of material structures, ultimately improving the prediction accuracy of the model.

Description

Technical Field

[0001] This invention relates to the field of materials prediction and analysis technology, and more specifically to a method, system and device for predicting materials properties based on graph neural networks. Background Technology

[0002] Currently, machine learning methods have become an important tool for predicting material properties and designing new materials. Accurate description of material structure is one of the key factors in predicting material properties. In terms of material structure characterization, methods such as feature sets based on human experience and polynomial expansions based on atomic environments are no longer sufficient to meet the accuracy requirements for predicting material properties. Graph neural networks, as a way to intuitively extract structural features, are increasingly being applied to the characterization of crystal structures, such as the Crystal Graph Convolutional Neural Networks (CGCNN) framework. However, simple graph representations containing only point and edge information cannot accurately represent the global information of the material structure, thus affecting the accuracy of predicted material properties.

[0003] To address the problems in the relevant technologies, no effective solutions have yet been proposed. For example, the data-driven material property prediction method and system disclosed in application number 202110705753.2 only involves initial features and combinations of features. These combined features cannot fully and accurately characterize the material structure, especially lacking the characterization of crystal bond angle information, thus affecting the accuracy of predicting material properties.

[0004] Therefore, how to provide a material property prediction method, system, and device based on graph neural networks that can accurately represent global information of material structure is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a material property prediction method, system and device based on graph neural networks, which can realize global information representation through atomic local environment. In this method, graph neural networks are used to represent the material graph structure information. In addition to node and edge information, the structure representation also includes edge update and angular information of three-body interaction. The representation of the entire structure is extended through three layers of convolution to obtain an accurate representation of the material structure, thereby improving the prediction accuracy of the model.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for predicting material properties based on graph neural networks includes the following steps:

[0008] Obtain the crystal structure of the material;

[0009] The crystal structure is encoded, initialized, and normalized to obtain a graph network of corresponding atoms;

[0010] Construct a structural feature prediction model, determine the network hyperparameter information of the structural feature prediction model, and set the network evaluation algorithm, activation function, and optimizer of the structural feature prediction model;

[0011] The structural feature prediction model is trained and optimized based on the network hyperparameter information, the network evaluation algorithm, the activation function, and the optimizer.

[0012] The material data to be predicted is input into the structural feature prediction model after training and optimization, and the material property prediction results are output.

[0013] Preferably, the initialization and normalization of the crystal structure to obtain an initial dataset includes:

[0014] The required atomic properties are one-hot encoded according to their dimension.

[0015] For continuous point values, categories are divided according to a preset range and features are encoded accordingly;

[0016] For discrete point values, they are directly encoded according to the category;

[0017] Edge vectors are encoded with a fixed dimension based on the distance between atoms;

[0018] Initial feature vectors of multiple types of atoms and initial feature vectors of multiple types of edges are obtained respectively;

[0019] Combined into a graph network of corresponding atoms.

[0020] Preferably, the constructed structural feature prediction model includes:

[0021] The L1 hidden layer, the first graph convolutional layer, the second graph convolutional layer, the third graph convolutional layer, the pooling layer, and the L2 hidden layer are connected in sequence.

[0022] The L1 hidden layer maps all the initial feature vectors of the graph network into inputs suitable for the first graph convolutional layer, the second graph convolutional layer, and the third graph convolutional layer.

[0023] By extracting point values ​​and edge vectors from the first graph convolutional layer, the second graph convolutional layer, and the third graph convolutional layer, a graph convolutional layer model representing the crystal structure is obtained.

[0024] After passing through the first graph convolutional layer, the second graph convolutional layer, and the third graph convolutional layer, the graph convolutional layer model is pooled. The pooling layer synthesizes the local environments represented by different atoms into a graph vector representing the overall crystal environment.

[0025] Using normalized crystal vectors, material property prediction modeling is performed based on the L2 hidden layer, and the predicted property values ​​are output.

[0026] Preferably, the extraction of point values ​​and edge vectors through the first graph convolutional layer, the second graph convolutional layer, and the third graph convolutional layer is achieved through updating, specifically including:

[0027] The node vector is updated using the following formula:

[0028]

[0029] In the formula, W1 and W2 are training weights, b1 and b2 are biases, k represents the edge connecting two atoms, σ is a standard gating mechanism used to learn the parameters of the interaction between atoms, z is the superposition of the atom feature vector with the feature vectors of all adjacent nodes and the feature vectors of the connecting edges, and ⊙ represents the element-wise multiplication of matrices. The edge vector embedding update is specifically formulated as follows:

[0030]

[0031] In the formula, the parameters are defined as above.

[0032] The node vectors and edge vectors connecting atoms (i,j,l) are connected to form:

[0033]

[0034] In the formula, v is the feature vector of the point, and u is the feature vector of the edge. This represents the direct addition of vectors.

[0035] Preferably, the network hyperparameter information of the structural feature prediction model includes: the number of training samples per session, learning rate, number of training rounds, weight decay, number of graph convolutional layers, dimension of the input features of the material, cutoff radius, and number of maximum neighbor nodes.

[0036] Preferably, the network evaluation algorithm is the mean absolute error (MAE), the activation function is the Tanh function, and the optimizer is the Adamw optimizer.

[0037] Preferably, the step of training and optimizing the structural feature prediction model based on the network hyperparameter information specifically includes:

[0038] Based on the set network hyperparameter information, a network model is generated and trained. The set evaluation algorithm is used to continuously adjust the hyperparameter information to obtain the parameters corresponding to the optimal training model. The parameters generate corresponding structural feature vectors, and the generated structural feature vectors are used to predict the target properties.

[0039] A material property prediction system based on graph neural networks includes:

[0040] Acquisition module: Acquires the crystal structure of the material;

[0041] A preprocessing module, connected to the acquisition module, is used to encode, initialize, and normalize the crystal structure to obtain a graph network of corresponding atoms;

[0042] The construction module, connected to the preprocessing module, is used to construct a structural feature prediction model, determine the network hyperparameter information of the structural feature prediction model, and set the network evaluation algorithm, activation function, and optimizer of the structural feature prediction model.

[0043] The training and optimization module, connected to the construction module, is used to train and optimize the structural feature prediction model based on the network hyperparameter information, the network evaluation algorithm, the activation function, and the optimizer.

[0044] The prediction module, connected to the training and optimization module, is used to input the chemical reaction data to be predicted into the structural feature prediction model after training and optimization, and output the material structure characterization results.

[0045] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the material property prediction method based on a graph neural network as described above.

[0046] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a material property prediction method, system, and device based on graph neural networks, realizing the overall process of structure extraction and performance prediction based on graph neural networks. This includes the selection and construction of a material database, feature initialization of atoms and bonds in each data crystal, graph convolutional layer interaction of crystal structure information, pooling of all features to obtain new feature vectors and predicting crystal property values, comparing the error with the true property values ​​and backpropagating to optimize model parameters, ultimately obtaining a high-precision material property prediction model.

[0047] This invention introduces the interaction vector of bond angles between three bodies in crystal structure characterization, enabling more precise capture of implicit structural information and resulting in more accurate crystal structure characterization, thereby obtaining more accurate property predictions. This invention is applicable to the description of all material systems, solving the difficulty of inconsistent characterization between crystal structure and small molecule structure in materials. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0049] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0050] Figure 2 This is a schematic diagram of the structural feature prediction model of the present invention;

[0051] Figure 3 This is a graphical convolutional layer model representing the crystal structure of the present invention;

[0052] Figure 4 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] See appendix Figure 1 As shown, embodiments of the present invention disclose a method, system, and device for predicting material properties based on graph neural networks.

[0055] Includes the following steps:

[0056] Obtain the crystal structure of the material;

[0057] The crystal structure is encoded, initialized, and normalized to obtain the graph network of the corresponding atoms;

[0058] Construct a structural feature prediction model, determine the network hyperparameter information of the structural feature prediction model, and set the network evaluation algorithm, activation function, and optimizer of the structural feature prediction model;

[0059] The structural feature prediction model is trained and optimized based on network hyperparameter information, network evaluation algorithm, activation function, and optimizer.

[0060] The material data to be predicted is input into the structural feature prediction model after training and optimization, and the material property prediction results are output.

[0061] In one specific embodiment, material data for the desired predicted properties are collected, including crystal structure information and predicted property information. The data source can be self-built data or publicly available data. The dataset is divided into two parts: a training set for training the model and a validation set for validating the model's performance.

[0062] Specifically, obtaining the crystal structure of the materials involved: acquiring data from open-source materials projects, and after extraction and screening, using datasets that included various inorganic crystals ranging from simple metals to complex minerals. After removing poorly converging crystals, the entire database contained 23,744 material data entries, including 87 elements, 7 crystal systems, and 216 space groups.

[0063] In one specific embodiment, the crystal structure is encoded, initialized, and normalized to obtain an initial dataset, including:

[0064] The required atomic properties are one-hot encoded according to their dimension.

[0065] For continuous point values, categories are divided according to a preset range and features are encoded accordingly;

[0066] For discrete point values, they are directly encoded according to the category;

[0067] The edge vectors are encoded with a fixed dimension based on the distance between atoms, and this dimension can be adjusted as needed.

[0068] Initial feature vectors of multiple types of atoms and initial feature vectors of multiple types of edges are obtained respectively;

[0069] Combined into a graph network of corresponding atoms.

[0070] Specifically, the input data feature vector is initialized with the following specifications, starting from the first bit: 19 bits for the group number, 7 bits for the period number, 10 bits for the covalent radius, 10 bits for electronegativity, 10 bits for the first ionization energy, 4 bits for each orbital (s, p, d, f), 12 bits for the atomic volume, and 10 bits for the atomic distance. This results in a total of 92 bits of atomic input feature encoding.

[0071] In one specific embodiment, constructing a graph neural network to obtain structural feature models mainly involves providing hyperparameter information for the model network. The required hyperparameter information includes the number of training samples per iteration, learning rate, number of training epochs, weight decay, number of graph convolutional layers, dimension of the input features, cutoff radius, and number of maximum neighbor nodes. Then, a suitable activation function is determined, followed by a suitable optimizer function to adjust the model parameters to minimize the loss function, and a suitable network evaluation algorithm is determined.

[0072] For details, please see the appendix. Figure 2As shown, the constructed structural feature prediction model includes:

[0073] The L1 hidden layer, the first graph convolutional layer, the second graph convolutional layer, the third graph convolutional layer, the pooling layer, and the L2 hidden layer are connected in sequence.

[0074] The L1 hidden layer maps all the initial feature vectors of the graph network into inputs suitable for the first, second, and third graph convolutional layers.

[0075] By extracting point values ​​and edge vectors from the first, second, and third graph convolutional layers, a graph convolutional layer model representing the crystal structure is obtained.

[0076] After passing through the first, second, and third graph convolutional layers, the graph convolutional layer model is pooled. The pooling layer synthesizes the local environments represented by different atoms into a graph vector representing the overall crystal environment.

[0077] Using normalized crystal vectors, material property prediction modeling is performed based on the L2 hidden layer, and the predicted property values ​​are output.

[0078] In one specific embodiment, a graph structure description method incorporating nodes, edges, and three-body interactions is proposed to achieve crystal structure description including atoms, bond lengths, and bond angles. In this method, in addition to including node and edge information similar to existing methods, the explicit three-body correlation of adjacent constituent atoms and the optimized chemical representation of interatomic bonds are accurately expressed. The graph convolution update formula for nodes is as follows:

[0079]

[0080] In the formula, W1 and W2 are training weights, b1 and b2 are biases, k represents the edge connecting two atoms, σ is a standard gating mechanism used to learn the parameters of the interaction between atoms, z is the superposition of the atom feature vector, the feature vectors of all adjacent nodes, and the feature vectors of the connecting edges, and ⊙ represents the element-wise multiplication of matrices.

[0081] The above formula describes the vector update relationship between the three atoms and the two edges, explicitly adding the description relationship of the angular vectors between the three bodies.

[0082] The following addresses the issue of edge vector embedding updates, which better represent the local environment of the crystal:

[0083]

[0084] The above formula describes the vector update relationship between the three atoms and the two edges, explicitly adding the description relationship of the angular vectors between the three bodies;

[0085] The node vectors and edge vectors connecting atoms (i,j,l) are connected to form:

[0086]

[0087] In the formula, v is the feature vector of the point, and u is the feature vector of the edge. This represents the direct addition of vectors.

[0088] Specifically, in the structural feature prediction model, the local environment of atoms in the crystal structure is better represented by updating the edge vectors of each node by connecting it to its neighboring nodes. Furthermore, this model can utilize crystal structure information and inter-atomic distance information in the edge vector embedding of the crystal graph.

[0089] For details, please see the appendix. Figure 3 As shown, this embodiment provides a graph convolutional layer model for characterizing crystal structures. The model maintains a point vector for each node and an edge vector for each edge. The graph convolutional layer aggregates the feature information between nodes and between nodes and edges, and selects the 12 atoms with the smallest distance as neighboring nodes. Feature vectors are extracted for each atom and edge of each different property in the crystal structure graph. A local environment connectivity graph is constructed on each atom. The atomic graph is updated through an L1 hidden layer (fully connected layer) and several graph convolutional layers. At this point, the feature vector of each atom represents its local environment information. Because the number of atoms in each crystal is finite, the update of the three-layer graph convolutional vector can almost extend to the entire graph. Therefore, the method of adding the characteristics of inter-atomic bond angles and bond length features to the structural characterization in this invention is of great significance for better capturing the implicit information in the crystal structure and achieving accurate prediction of material property values.

[0090] In one specific embodiment, the network hyperparameter information of the structural feature prediction model includes: the number of training samples per session, learning rate, number of training rounds, weight decay, number of graph convolutional layers, dimension of the input features of the material, cutoff radius, and number of maximum neighbor nodes.

[0091] Specifically, based on the initial dataset, the network hyperparameters are designed. The number of training samples per run is set to 128, the learning rate is 0.001, the number of training epochs is 300, the weight decay is 1e-05, the number of graph convolutional layers is three, the dimension of the input features of the material is 92, the truncation radius is 8, and the maximum number of neighboring nodes is 12.

[0092] In one specific embodiment, the network evaluation algorithm is the mean absolute error (MAE), the activation function is the Tanh function, and the optimizer is the Adamw optimizer.

[0093] In one specific embodiment, the structural feature prediction model is trained and optimized based on network hyperparameter information, specifically including:

[0094] Based on the set network hyperparameter information, a network model is generated and trained. The set evaluation algorithm is used to continuously adjust the hyperparameter information to obtain the parameters corresponding to the optimal training model. The parameters generate corresponding structural feature vectors, and the generated structural feature vectors are used to predict the target properties.

[0095] Specifically, the training and optimization of the graph neural network model are based on the network hyperparameter information set above. A network model is generated for training, and structural feature vectors corresponding to different weights are obtained. The evaluation algorithm set above is used to predict the target property using the structural feature vectors; in this embodiment, the target property is the material's formation energy. The predicted target and the actual target are evaluated using the set model evaluation algorithm. Backpropagation is then performed based on these evaluation values ​​to further optimize the network model parameters, ultimately obtaining the network model with the most accurate prediction performance. The hyperparameters and corresponding weights of this network model, along with the evaluation algorithm used by the network model, are stored in the model database.

[0096] More specifically, in the process of improving the overall model, after obtaining the material structure features, the features are input into a multi-layer ordinary neural network to predict the attribute values, and an evaluation algorithm is used to evaluate the direct error between the predicted values ​​and the true values ​​of the constructed model. After backpropagation and multiple training sessions, an accurate prediction model is obtained.

[0097] In one specific embodiment, the application model is used to predict material properties, specifically: based on material crystal data generated by other means, the model is used to predict attribute values, and various analyses and judgments are performed on the material data based on the prediction results.

[0098] On the other hand, see appendix Figure 4 As shown, the present invention also discloses a material property prediction system based on graph neural networks, comprising:

[0099] Acquisition module: Acquires the crystal structure of the material;

[0100] The preprocessing module, connected to the acquisition module, is used to encode, initialize, and normalize the crystal structure to obtain the graph network of the corresponding atoms;

[0101] The construction module, connected to the preprocessing module, is used to build the structural feature prediction model, determine the network hyperparameter information of the structural feature prediction model, and set the network evaluation algorithm, activation function, and optimizer of the structural feature prediction model.

[0102] The training and optimization module, connected to the construction module, is used to train and optimize the structural feature prediction model based on network hyperparameter information, network evaluation algorithms, activation functions, and optimizers.

[0103] The prediction module, connected to the training and optimization module, is used to input the chemical reaction data to be predicted into the structural feature prediction model after training and optimization, and output the material structure characterization results.

[0104] In another aspect, the present invention also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the material property prediction method based on graph neural networks as described above.

[0105] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0106] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A material property prediction method based on a graph neural network, characterized in that, The method comprises the following steps: obtaining a crystal structure of a material; performing encoding initialization and standardization processing on the crystal structure to obtain a graph network corresponding to atoms; constructing a structure feature prediction model, determining network hyperparameter information of the structure feature prediction model, and setting a network evaluation algorithm, an activation function, and an optimizer of the structure feature prediction model; training and optimizing the structure feature prediction model based on the network hyperparameter information and the network evaluation algorithm, the activation function, and the optimizer; inputting material data to be predicted into the structure feature prediction model after training and optimization, and outputting a material property prediction result; The constructed structure feature prediction model comprises: an L1 hidden layer, a first graph convolution layer, a second graph convolution layer, a third graph convolution layer, a pooling layer, and an L2 hidden layer connected in sequence; The L1 hidden layer maps all initial feature vectors of the graph network into inputs suitable for the first graph convolution layer, the second graph convolution layer, and the third graph convolution layer; The first graph convolution layer, the second graph convolution layer, and the third graph convolution layer extract information of point values and edge vectors to obtain a graph convolution layer model representing the crystal structure; After the first graph convolution layer, the second graph convolution layer, and the third graph convolution layer, the graph convolution layer model is pooled, and the local environment represented by different atoms is synthesized into a graph vector representing the overall crystal environment through the pooling layer; The normalized crystal vector is used to perform material attribute prediction modeling based on the L2 hidden layer, and a predicted attribute value is output.

2. The material property prediction method based on a graph neural network according to claim 1, characterized in that, The encoding initialization and standardization processing on the crystal structure to obtain an initial data set comprises: one-hot encoding of the required atomic attributes according to the number of dimensions, wherein, for continuous point values, classes are divided according to a preset range, and features are encoded respectively; for discrete point values, encoding is directly performed according to the classes; for edge vectors, fixed-dimension encoding is performed according to the distance between atoms; initial feature vectors of multiple types of atoms and initial edge feature vectors of multiple types are obtained respectively; a graph network corresponding to the atoms is combined.

3. The material property prediction method based on a graph neural network according to claim 1, characterized in that, The extraction of information of point values and edge vectors through the first graph convolution layer, the second graph convolution layer, and the third graph convolution layer is implemented through updating, and specifically comprises: node vector updating, specifically in the formula: ； wherein, W1 and W2 are training weights, b1 and b2 are biases, k represents an edge connecting two atoms, σ is a standard gating mechanism used to learn the parameters of the interaction between atoms, z is the superposition of atomic feature vectors and all adjacent node feature vectors and the feature vectors of connecting edges, and represents matrix element-wise multiplication; edge vector embedding updating, specifically in the formula: ； The node vector and the edge vector of the connecting atom (i, j, l) are connected to form: ； wherein, v is the feature vector of a point, u is the feature vector of an edge, and represents vector addition.

4. The material property prediction method based on a graph neural network according to claim 1, characterized in that, The network hyperparameter information of the structure feature prediction model comprises: the number of single training samples, the learning rate, the number of training rounds, the weight decay, the number of graph convolution layers, the dimension of the input features of the material, the cutoff radius, and the maximum number of neighbor nodes.

5. The material property prediction method based on graph neural network according to claim 1, wherein, The network evaluation algorithm is mean absolute error (MAE), the activation function is a Tanh function, and the optimizer is an Adamw optimizer.

6. The material property prediction method based on a graph neural network according to claim 1, characterized in that, The training and optimization of the structure feature prediction model based on the network hyperparameter information specifically include: Based on the set network hyperparameter information, a network model is generated and trained to obtain structure feature vectors corresponding to different weights, and a target property is predicted using a structure feature vector by using a set evaluation algorithm.

7. A graph neural network-based material property prediction system using the graph neural network-based material property prediction method according to any one of claims 1-6. It includes: An acquisition module acquires a crystal structure of a material; A preprocessing module, connected to the acquisition module, is configured to perform encoding initialization and standardization processing on the crystal structure to obtain a graph network corresponding to atoms; A construction module, connected to the preprocessing module, is configured to construct a structure feature prediction model, determine network hyperparameter information of the structure feature prediction model, and set a network evaluation algorithm, an activation function, and an optimizer for the structure feature prediction model; A training and optimization module, connected to the construction module, is configured to train and optimize the structure feature prediction model based on the network hyperparameter information and the network evaluation algorithm, the activation function, and the optimizer; A prediction module, connected to the training and optimization module, is configured to input to-be-predicted chemical reaction data into the structure feature prediction model that has been trained and optimized to output a material structure representation result.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the material property prediction method based on a graph neural network according to any one of claims 1 to 6.

Images