Rare earth compound prediction method based on pre-training fine-tuning graph neural network

Abstract

The invention belongs to the field of chemical informatization, provides a rare earth compound prediction method based on a pre-training fine-tuning graph neural network, and aims to solve the problem of low prediction efficiency caused by combination explosion and complex properties. According to the main scheme, binary compound combinations of rare earth elements and other elements are generated through permutation and combination, and screening is conducted according to chemical rules and a known compound database to obtain a candidate rare earth compound set; converting each compound in the candidate rare earth compound set into graph structure data comprising node features and edge features, wherein the node features comprise rare earth mark variables used for marking rare earth elements; inputting the graph structure data into a pre-trained fine-tuned graph neural network model to output prediction formation energy and prediction feasibility probability of each candidate rare earth compound; and calculating a comprehensive score of each candidate rare earth compound according to the prediction formation energy and the prediction feasibility probability, and sorting and screening the candidate rare earth compounds according to the comprehensive scores.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of chemical information, and provides a rare earth compound prediction method based on a pre-trained fine-tuning graph neural network. BACKGROUND

[0002] Due to the unique 4f5d6s valence electron configuration and outstanding coordination bonding ability, rare earth elements have excellent magnetic, optical, electrical, catalytic and biomedical properties, and are core functional materials supporting the development of strategic fields such as modern high-end manufacturing, new energy and national defense technology. The innovative research and development of their compounds is directly related to the core competitiveness of the industry chain. With the surge in global demand for high technology such as national defense security, information, new energy, high-end equipment and quantum technology, existing rare earth compounds have been difficult to meet the needs of the industry, and the efficient discovery of new rare earth compounds has become a core task to break through technical bottlenecks.

[0003] Traditional methods are centered on chemical experience, and are based on chemical experience (such as element valence matching and analogy of similar compounds) to design a synthesis scheme. These methods have the disadvantages of long cycle (a single experiment takes several days to several months), high cost (large investment in consumables and equipment), and limited coverage (unable to systematically explore the vast element combination space), making it difficult to break through the limitations of experience and discover new compounds.

[0004] Computational chemistry methods based on density functional theory (DFT) predict compound stability by simulating electronic structures, which is currently the mainstream tool for theoretical prediction. However, DFT calculations require the pre-input of the crystal structure (such as lattice parameters and atomic coordinates) of the compound, and the structural information of unknown rare earth compounds is difficult to obtain in advance. For example, for the “La-Al-Ce” ternary system, if there is no known structure template, thousands of possible crystal configurations need to be traversed, and the calculation cost increases exponentially, limiting its application in large-scale candidate compound screening. SUMMARY

[0005] The present application aims to solve the problem of efficiently and accurately predicting the stability and feasibility of unknown rare earth compounds caused by the huge combination space of rare earth compounds and the complexity of their chemical properties.

[0006] To achieve the above purpose, the present application adopts the following technical means:

[0007] The present application provides a rare earth compound prediction method based on a pre-trained fine-tuning graph neural network, comprising the following steps:

[0008] Step S1: Generate a set of candidate rare earth compounds: generate binary compound combinations of rare earth elements and other elements through permutation and combination, and screen them according to chemical rules and known compound databases to obtain a set of candidate rare earth compounds;

[0009] Step S2: Compound graph embedding: converting each compound in the candidate rare earth compound set into graph structure data, which includes node features and edge features, wherein the node features contain a rare earth flag variable for identifying rare earth elements;

[0010] Step S3: Compound property prediction: inputting the graph structure data into a pre-trained and fine-tuned graph neural network model, which outputs the predicted formation energy and the predicted feasibility probability of each candidate rare earth compound;

[0011] Step S4: Compound screening: calculating the comprehensive score of each candidate rare earth compound according to the predicted formation energy and the predicted feasibility probability, and ranking and screening the candidate rare earth compounds according to the comprehensive score.

[0012] In the above scheme, the screening according to chemical rules in step S1 includes:

[0013] excluding combinations of rare earth elements and noble gases; and / or

[0014] excluding compound combinations that do not satisfy the valence balance principle, wherein the valence balance principle is that the sum of the oxidation states of the elements in the compound is zero.

[0015] In the above scheme, step S1 includes:

[0016] Step S11: generating compound combinations: generating all possible binary compound combinations by combining the rare earth element set with the elements in the other element set in pairs through permutation and combination, wherein:

[0017] the rare earth element set is {La, Ce, Nd, Sm, Eu, Gd, Tb, Dy, Ho, Er, Tm, Yb, Lu};

[0018] the other element set is {O, N, S, C, Al, Fe, Ca, Ba};

[0019] the compound combinations are generated by the following formula:

[0020]

[0021] wherein, and represent the indices of rare earth elements and other elements, respectively;

[0022] Step S12: rule-based screening: screening the compound combinations to exclude combinations that do not conform to scientific laws, specifically including: excluding combinations of rare earth elements and noble gases, i.e.

[0023] wherein, and represent the index of rare earth elements and noble gases respectively; exclude combinations that do not meet the valence balance principle, that is:

[0024]

[0025] wherein, and represent the element and the element respectively;

[0026] Step S13: Screening of chemical database: for the remaining compound combinations after rule screening, by querying the chemical database, exclude the existing compounds, that is:

[0027]

[0028] wherein, f(compound combination) refers to whether the compound exists in the database returned by API query.

[0029] In the above scheme, step S2 includes:

[0030] Step S21: Building a graph structure: representing the candidate rare earth compound as an undirected graph wherein is a node set, representing the atoms in the compound, is an edge set, representing the bonding relationship between atoms, the construction of the edge is based on the element valence and the valence balance principle, and only establishes edges between rare earth elements and non-rare earth elements;

[0031] Step S22: Building node features: for each node build a node feature vector wherein is the atomic number of the element, is the electronegativity of the element, is the atomic radius of the element, is the rare earth element flag variable, when the element is a rare earth element , otherwise ;

[0032] Step S23: Building edge features: for each edge build an edge feature vector wherein, is the absolute value of the electronegativity difference between node and node , is the atomic radius difference between node with the node the absolute value of the atomic radius difference;

[0033] Step S24: rare earth node weighting: for the node of the rare earth element flag variable , the characteristic vector is weighted and amplified to obtain the updated node characteristic vector , wherein is a rare earth node enhancement coefficient greater than 1;

[0034] Step S25: generating a graph representation: constructing a node feature matrix according to the updated characteristic vectors of all nodes , and constructing an adjacency matrix according to the edge relationship between nodes , wherein if there is an edge , then 1, otherwise , finally obtaining the graph structure of the candidate rare earth compound .

[0035] In the above scheme, in the step S3, the training method of the pre-trained fine-tuned graph neural network model comprises:

[0036] Pre-training phase: using a general compound dataset that does not contain rare earth elements, training the graph neural network model to learn to predict the formation energy and feasibility of the compound;

[0037] Fine-tuning phase: using a known rare earth compound dataset, fine-tuning the parameters of the model obtained by pre-training to make the model adapt to the characteristics of rare earth compound prediction.

[0038] In the above scheme, step S3 comprises:

[0039] Step S31: model construction: constructing a double-output structure model based on a graph attention network (GAT), the model taking the graph structure data of the compound as input, extracting features through multiple layers of GAT, and then predicting the formation energy and feasibility probability of the compound through two parallel output branches respectively;

[0040] Step S32: model pre-training: using a general compound dataset that does not contain rare earth elements to pre-train the model, the goal of the pre-training is to simultaneously optimize the formation energy regression task and the feasibility classification task;

[0041] Step S33: model fine-tuning: using a known rare earth compound dataset, fine-tuning the model based on the pre-training completed model, the fine-tuning process freezes the graph neural network layer parameters in the model except the last layer, and only updates the parameters of the last layer of graph neural network layer and the two output branches;

[0042] Step S34: Joint loss function optimization: In the pre-training and fine-tuning stages, a weighted joint loss function is adopted The model is optimized, and the joint loss function is defined as:

[0043]

[0044] wherein, and is a hyperparameter for balancing the importance of regression and classification tasks; is the mean square error loss for the regression task of forming energy, defined as:

[0045]

[0046] wherein, B is the batch size, is the predicted formation energy of the i-th sample, is the true formation energy of the i-th sample; Lfeas is the binary cross-entropy loss of the feasibility classification task, defined as:

[0047]

[0048] wherein, is the true feasibility label {0 or 1} of the i-th sample, is the feasibility probability predicted by the model for the i-th sample;

[0049] Step S35: Prediction execution: input the graph structure data of the rare earth compound to be tested into the pre-trained and fine-tuned model to obtain the predicted formation energy and predicted feasibility probability of the compound.

[0050] In the above scheme, a three-layer graph attention network is used for node feature encoding. In the graph neural network encoding stage, the main task of each layer of GAT is to fuse the local neighbor information of the node to generate node embedding, so that the node features can reflect the structure and element properties of the compound graph. The processing flow of each layer is as follows:

[0051] a) Linear mapping: for each node i in the l-th layer and the k-th attention head, the node feature is mapped to a low-dimensional space by linear transformation to obtain the mapped node feature ; and the edge feature is linearly mapped to obtain the mapped edge feature ;

[0052] The mapping process is as follows:

[0053]

[0054] ​

[0055] where, and are learnable weight matrices, is the dimension of the input features of the l-th layer, is the dimension of the attention head, is the dimension of the edge features, denotes the transpose;

[0056] b) Computing attention scores: for node i and its neighbor node j, compute the attention score , which considers both the mapped node features and edge features;

[0057] The computing process is as follows:

[0058]

[0059] where, is a learnable vector, is an activation function, denotes the vector concatenation operation;

[0060] c) Attention normalization and Dropout: normalize the attention scores of all neighbors of node i, and apply Dropout to obtain the normalized attention weights , and the normalization and Dropout process is as follows:

[0061]

[0062] where, N(i)N(i) is the set of neighbor nodes of node ii, is the probability of Dropout;

[0063] d) Node aggregation and update: according to the normalized attention weights, perform weighted summation on the mapped features of the neighbor nodes, and update the node features through a nonlinear activation function to obtain the output of the k-th attention head , and the updating process is as follows:

[0064]

[0065] e) Multi-head merging and residual connection: for the GAT of the non-last layer, concatenate the outputs of all attention heads, and combine the residual connection and layer normalization to obtain the input of the next layer , and the merging process is as follows:

[0066]

[0067] where K is the number of attention heads, LayerNorm is the layer normalization operation, is the residual projection operation;

[0068] f) Final layer aggregation: for the last layer GAT, the outputs of all attention heads are averaged and combined with a residual connection and layer normalization to obtain the final node embeddings The final aggregation process is as follows:

[0069]

[0070] where K is the number of attention heads in the last layer.

[0071] In the above scheme, after the graph attention network encoding is completed, the node embeddings are converted into graph-level features, and a double-task prediction is performed through a fully connected layer, which includes the following steps:

[0072] a) Graph-level feature pooling: the node embeddings obtained after graph neural network encoding are pooled to generate a graph-level feature vector The pooling is average pooling, and the calculation formula is:

[0073]

[0074] where N is the total number of nodes in the graph, is the embedding representation of the i-th node in the last layer;

[0075] b) Formation energy regression: the graph-level feature vector is input into a formation energy regression branch to predict the formation energy of the compound through a fully connected layer The prediction process is:

[0076]

[0077] where and are the weight matrix and bias term of the formation energy regression branch, respectively;

[0078] c) Feasibility classification prediction: the graph-level feature vector is input into a feasibility classification branch in parallel to predict the feasibility probability of the compound through a fully connected layer The prediction process is:

[0079] where and are the weight matrix and bias term of the feasibility classification branch, respectively.

[0080] ​In the above scheme, the model fine-tuning in step s33 comprises the following steps:

[0081] Step s331: freeze the parameters of the graph neural network layers except the last layer in the pre-trained model, and only update the parameters of the last layer graph attention network (GAT) and two fully connected prediction heads to prevent catastrophic forgetting;

[0082] Step s332: define the parameter set to be updated as where GAT(3) is the parameter of the last layer GAT, and are the weights and biases of the formation energy prediction branch, and are the weights and biases of the feasibility prediction branch;

[0083] Step s333: take the minimization of the loss function LL as the optimization objective, and solve the optimal parameter set , and the optimization objective is defined as:

[0084]

[0085] where is the optimal parameter set, and L is the loss function;

[0086] Step s334: use the Adam optimizer for optimization, set the learning rate to 1×10 −4 , the batch size is 16, and the early stopping strategy is used in the training process. If the loss does not decrease for 10 consecutive rounds, the training is stopped.

[0087] In the above scheme, step S4 comprises:

[0088] Obtaining the predicted formation energy and predicted feasibility probability of the rare earth compound to be tested;

[0089] According to the predicted formation energy and predicted feasibility probability, calculating the comprehensive score of each rare earth compound to be tested :

[0090]

[0091] wherein, is the comprehensive score of the i th compound, is the predicted formation energy of the i th compound, is the predicted feasibility probability of the i th compound, is the maximum predicted formation energy among all rare earth compounds to be tested, is the minimum predicted formation energy among all rare earth compounds to be tested, is a small value to prevent division by zero; ​

[0092] According to the comprehensive score ) to sort all rare earth compounds to be tested, and screen the compound with a higher comprehensive score as a potential viable rare earth compound.

[0093] Because the present application adopts the above technical means, it has the following beneficial effects:

[0094] 1. The present application solves the technical problems that the traditional method is difficult to systematically explore massive element combinations and is prone to produce invalid or repeated compound candidates by the multi-stage screening technical means of step S1 (arranging and combining binary compound combinations, excluding unreasonable combinations based on chemical rules, and querying a chemical database to remove known compounds), achieving the effect of quickly generating a candidate rare earth compound set that conforms to scientific laws and has innovation, providing high-quality input for subsequent prediction, and reducing invalid experimental exploration costs.

[0095] 2. The present application solves the technical problems that the existing method needs to rely on complete crystal structure to characterize the compound and cannot highlight the core role of rare earth elements by the rare earth-enhanced graph embedding technical means of step S2 (constructing node features containing rare earth marker variables, edge features based on element attribute differences, and rare earth node weighting amplification), achieving the effect of realizing effective characterization of compounds and accurately capturing the unique chemical properties of rare earth elements without complete crystal structure, reducing the dependence on experimental structure data.

[0096] 3. The present application solves the technical problems of insufficient generalization ability of a single model and low prediction accuracy of rare earth compound characteristics by the pre-training-fine-tuning dual-task model technical means of step S3 (pre-training learning general chemical rules based on ordinary compound data set, fine-tuning adaptation of special properties based on rare earth compound data set, and simultaneous prediction of formation energy and feasibility probability based on GAT dual-output branch), achieving the effect of balancing model universality and specificity, and significantly improving the prediction accuracy of formation energy and feasibility.

[0097] 4. The present application organically combines the technical means of multi-stage precise screening (S1), rare earth-enhanced graph embedding characterization (S2), pre-training-fine-tuning dual-task prediction (S3), and comprehensive score targeted screening (S4), achieving the effect of whole-process collaborative optimization. Each technical link is closely linked, the previous step provides high-quality input for the subsequent step, and the subsequent step realizes accurate judgment based on the previous result, together solving the core problems of long research and development cycle, high cost, and limited coverage of the traditional method, and finally realizing the efficient and accurate discovery of new rare earth compounds, providing clear experimental direction for researchers, greatly shortening the research and development cycle and reducing experimental costs. BRIEF DESCRIPTION OF DRAWINGS

[0098] Figure 1Flowchart of the present application;

[0099] Figure 2 Flowchart of the present application. DETAILED DESCRIPTION

[0100] The embodiments of the present application will be described in detail below. Although the present application will be described and illustrated by referring to specific embodiments, it is noted that the present application is not limited to these embodiments only. On the contrary, modifications or equivalent arrangements within the scope of the present application are to be encompassed by the claims.

[0101] In addition, in order to better illustrate the present application, numerous specific details are given in the following detailed description. Those skilled in the art will understand that the present application can also be implemented without these specific details.

[0102] The "pre-training-fine-tuning" graph neural network architecture proposed in this application innovatively combines chemical rule screening with machine learning, solving the key pain points of the prior art. Through a lightweight graph embedding method, effective characterization of compounds can be achieved without complete crystal structures, reducing data dependence; the rare earth node weighting mechanism and the targeted pre-training-fine-tuning strategy enable the model to accurately capture the unique chemical properties of rare earth elements, significantly improving prediction accuracy; the multi-stage screening process (permutation and combination generation, rule investigation, and database deduplication) ensures the scientificity and innovation of candidate compounds, providing high-quality input for subsequent experimental verification.

[0103] This method predicts the formation energy and feasibility of compounds through an end-to-end process, transforming the original "blind screening" experimental exploration into targeted verification, significantly shortening the research and development cycle and reducing experimental costs. The potential stable compounds screened can directly provide clear directions for experimental synthesis, helping researchers quickly break through the performance bottlenecks of existing rare earth materials and providing core technical support for the industrialization of high-performance rare earth-based functional materials.

[0104] The present application provides a rare earth compound prediction method based on a pre-training fine-tuning graph neural network, in order to facilitate those skilled in the art to better understand the technical concept of the present application, the following detailed description is made:

[0105] First part Generation and screening method of rare earth compounds to be tested

[0106] The method provided by the present application comprises the following three steps:

[0107] Step 1.1, list all possible binary compounds by permutation and combination:

[0108] First, list all possible combinations of rare earth compounds by permutation and combination, the specific steps are as follows:

[0109] List all rare earth elements, including lanthanides (La to Lu) and scandium (Sc), yttrium (Y), etc.

[0110]

[0111] List other elements that can be combined with rare earth elements, usually including: non-metallic elements such as oxygen (O), nitrogen (N), sulfur (S), carbon (C), fluorine (F), etc., and metallic elements such as aluminum (Al), iron (Fe), calcium (Ca), barium (Ba), etc.

[0112]

[0113] By combining each rare earth element with other elements in pairs, all possible binary compounds are generated. Similar combinations are made for all rare earth elements and other elements to generate all possible binary compound combinations. Compound pairs are generated by the following permutation formula:

[0114]

[0115] where, and represent the index of rare earth elements and other elements, respectively.

[0116] Step 1.2, rule-based screening:

[0117] After listing all possible binary compounds, rule-based screening is performed to exclude combinations that do not conform to scientific laws. The main screening rules include:

[0118] Inert gases (such as helium, neon, argon, etc.) are not easily involved in chemical reactions, so combinations of rare earth elements and inert gases cannot form stable compounds. Therefore, compound combinations of rare earth elements and inert gases should be excluded. The exclusion method is as follows:

[0119]

[0120] Each element has a specific oxidation state in a compound. If the oxidation states of the elements in the combination do not match, the combination cannot exist stably. For example, lanthanum (La) usually exists in +3 oxidation state, and lanthanum and oxygen usually form La2O3 when combined.

[0121]

[0122] where, oxidation state refers to the charge number of the element in the compound. For example, the oxidation state of oxygen is usually -2.

[0123] Step 1.3, screening of chemical database

[0124] After applying the rule screening, further known compound screening is performed on the remaining combinations to exclude compounds that have been discovered and synthesized. The purpose of this step is to ensure that the new compounds generated are innovative.

[0125] Query whether a compound already exists through a chemical database (such as PubChem, ChemSpider). Database queries can be implemented through corresponding API interfaces. For example, the molecular formula or compound name of a compound can be queried.

[0126] For compounds that already exist in the database, they are directly removed from the candidate combinations to ensure that only new compounds are retained. The query method is as follows:

[0127]

[0128] wherein, is to query through the API to return whether the compound already exists in the database.

[0129] Through the above three steps, this method can effectively generate and screen rare earth compound combinations that conform to scientific principles and have innovation. These compounds will be used as input for subsequent machine learning models to predict the formation energy and feasibility of the compounds, providing theoretical support for the discovery and application of new rare earth compounds.

[0130] Second part Compound graph embedding method

[0131] Step 2.1 Rare earth element embedding method to be tested

[0132] After the aforementioned permutation and combination, rule screening, and database screening, a series of candidate rare earth compounds that have not been synthesized or reported are obtained. In order to input these compounds into the graph neural network model for prediction, the molecular formula of the compound needs to be converted into a graph structure form.

[0133] The present application adopts a lightweight graph embedding method based on molecular formula, focusing on retaining key attributes that reflect the characteristics of rare earth elements to ensure that the model has both rare earth recognition ability and avoids excessive reliance on experimental property data that is difficult to obtain.

[0134] Each compound is represented as an undirected graph:

[0135]

[0136] wherein, is a set of nodes, representing atoms in the compound, is the edge set, representing the bonding relationship between atoms. Since there is only a molecular formula without a specific structure formula here, the construction of edges uses rules based on the balance of element valence and oxidation state, rather than accurate molecular geometry.

[0137] Each node represents an element atom in the compound. The node feature vector The embedding can reflect the core properties of rare earth elements, defined as follows:

[0138]

[0139] wherein is the atomic number of the element, used to distinguish element categories; is the electronegativity of the element, reflecting the bonding trend; is the atomic radius of the element, used to reflect the influence of spatial arrangement; is the rare earth element flag variable, when the element is a rare earth element = 1, otherwise = 0.

[0140] This design ensures that the model can identify and emphasize the special role of rare earth elements in the compound without relying on complete three-dimensional structure information.

[0141] The construction of edges is based on the principles of stoichiometry and oxidation state balance. For any element pair (A, B), if both appear in the molecular formula of the compound, an edge is established between them, and the edge feature vector is defined as:

[0142] wherein,

[0143] is the electronegativity difference, used to reflect the bond polarity; is the atomic radius difference, used to reflect the degree of spatial matching. This edge representation based on attribute differences can effectively simulate the relative relationship of bond strength under the condition of no structure information. For each compound, the node feature matrix and adjacency matrix are constructed:

[0144]

[0145]

[0146]

[0147] This representation makes the model input uniform as a data structure that can be processed by a graph neural network.

[0148] Considering the decisive role of rare earth elements in compounds, the invention introduces a rare earth node weighting mechanism in the graph input stage: for any rare earth node, when inputting the model, its feature vector is weighted and amplified: ​

[0149]

[0150] wherein is the rare earth node enhancement coefficient, used to amplify the model's attention to rare earth atoms, so that the model pays more attention to the influence of rare earth centers during feature propagation.

[0151] After the above steps, each candidate rare earth compound is uniformly converted to:

[0152]

[0153] The graph structure will be used as the input of the graph neural network, and after multi-layer graph attention propagation, the full graph representation is obtained, which is used for subsequent feasibility and formation energy prediction.

[0154] Step 2.2 Compound embedding method for training

[0155] First, collect the compound data for pre-training: the pre-training stage uses public chemical property databases, of which the Materials Project and OQMD are the main data sources. Both contain a large number of crystal structures and energy information of inorganic compounds, with complete structure files and formation energy data.

[0156] In order to control the data size and ensure the sample quality, the present application screens common compounds that meet the following conditions in the pre-training stage: no rare earth elements; containing 2-4 different elements; complete formation energy data; crystal structure information is analyzable. After screening, it is ensured that the model can learn sufficient structure-energy relationship, while avoiding training redundancy caused by too large data.

[0157] Then collect known rare earth compound data for fine-tuning the model: the fine-tuning stage uses known rare earth compound samples in public databases, including but not limited to Materials Project, ICSD and part of published literature data. The sample selection criteria are: containing at least one rare earth element; complete compound structure information or can be inferred; formation energy and stability label (whether can be experimentally synthesized) can be obtained. After screening, a certain number of known rare earth compound samples are obtained for the fine-tuning training stage of the model parameters.

[0158] The compound embedding method for training is similar to the rare earth element embedding method to be tested, except that the node embedding is added. Since the public compound data is comprehensive and structured, two elements are added to make the node embedding information more rich. The specific embedding method is as follows:

[0159]

[0160] wherein The atomic number of an element is used to distinguish element categories; Electronegativity of elements reflects their tendency to combine; The atomic radius of the element is used to reflect the influence of spatial arrangement. These three elements have the same intercalation as the rare earth compound being tested, while... For the number of valence electrons, The coordination number (obtained from crystal structure analysis).

[0161] The goal of graph embedding is to encode atomic-level structural information into graph vector representations for subsequent prediction tasks. Through graph attention mechanisms, the model can learn weighted dependencies between nodes, enabling the embedding results to simultaneously capture local bond features and the energy distribution characteristics of the overall crystal structure.

[0162] Part Three: Model Training

[0163] This model employs a dual-output structure based on a Graph Attention Network (GAT). Taking the graph representation of the compound as input, the model first receives the graph structure information of the compound from the input layer. Multiple layers of GAT extract the feature interactions and global representations between nodes. Finally, the two output branches predict the formation energy and feasibility probability respectively, thus achieving a joint assessment of energy stability and structural feasibility. The training process consists of two stages: first, pre-training on common compound data, followed by fine-tuning on known rare earth compounds to further capture the unique characteristics of rare earth systems. The entire training process uses a weighted joint loss function to simultaneously optimize the formation energy prediction error and feasibility classification error, thereby achieving end-to-end prediction of the feasibility of unknown rare earth compounds.

[0164] Step 3.1 Pre-training the model using common compounds

[0165] During the pre-training phase, a large amount of common compound data (excluding rare earth compounds) is used to train the model. The aim is to enable the model to learn chemical laws and intermolecular interactions, laying the foundation for subsequent fine-tuning with rare earth compounds.

[0166] The pre-training tasks are divided into two parts: a regression task to predict the formation energy of a compound, and a classification task to predict the feasibility (0 or 1) of a compound.

[0167] This method uses a Graph Attention Network (GAT) as a pre-trained model architecture. GAT can effectively handle graph-structured data and is particularly suitable for representation learning of molecular graphs.

[0168] This model uses a three-layer graph attention network (GAT) for node feature encoding. In the graph neural network encoding stage, the main task of each GAT layer is to fuse the local neighbor information of nodes to generate node embeddings, enabling the node features to reflect the structure and elemental properties of the compound graph. The processing flow for each layer is as follows:

[0169] Linear mapping: For each node, the original features or the output features from the previous layer are mapped to a lower dimensional space through a linear transformation, so that each attention head can learn different representations:

[0170]

[0171] For edge features (e.g., bond type, bond length), linear mapping is also done so that edge information can be included in attention calculation:

[0172]

[0173] where, and are learnable weight matrices, is the dimension of the input features of the l-th layer, is the dimension of the attention head, is the dimension of the edge features, denotes the transpose;

[0174] Calculate attention scores: For each node and its neighbor nodes attention scores are calculated to measure the importance of neighbors for node updates. Attention scores take into account both node features and edge features:

[0175]

[0176] where is a learnable vector, with a negative slope of 0.2, denotes the vector concatenation operation. This operation allows the network to learn "which neighbors are more important for the current node".

[0177] Attention normalization and Dropout

[0178] Normalize the attention scores of all neighbors of a node so that their weights sum to 1, and prevent overfitting through dropout during training:

[0179]

[0180] is the probability of Dropout;

[0181] Node aggregation and node update

[0182] Each node is updated by weighted summation of neighbor features:

[0183]

[0184] where is a nonlinear activation function. In this way, each node’s features incorporate neighbor information while preserving the node’s own features.

[0185] Multi-head merge with residual connection:

[0186] Concatenate the outputs of all attention heads, add a residual connection, and perform layer normalization:

[0187]

[0188] Take the average (instead of concatenation) of the multi-head outputs to obtain stable node embeddings:

[0189]

[0190] where K is the number of attention heads, LayerNorm is the layer normalization operation, is the residual projection operation, is the number of attention heads in the last layer;

[0191] Output the final node embeddings Directly used for downstream task prediction (formation energy regression and feasibility classification).

[0192] After GAT encoding, the node embeddings of each compound graph already contain local neighbor information and rare-earth element-specific properties. To predict the compound formation energy and feasibility, the node embeddings need to be converted into graph-level features (pooled) and passed through a fully connected layer for dual-task prediction.

[0193] Pooling: aggregate node embeddings into a graph vector using average pooling:

[0194]

[0195] as input to the fully connected layer.

[0196] Formation energy regression branch: predict the compound formation energy using a one-layer fully connected network :

[0197]

[0198] Feasibility classification branch: predict the compound feasibility probability using a one-layer fully connected network :

[0199]

[0200] and are the weight matrix and bias term of the formation energy branch respectively, are the weight matrix and bias term of the feasibility classification branch respectively;

[0201] Loss function: The total loss is constructed using the mean square error (MSE) of the regression task and the binary cross-entropy (BCE) of the classification task:

[0202]

[0203]

[0204] The total loss is a weighted average of the two:

[0205]

[0206] Here is a hyperparameter to balance the importance of the regression and classification tasks. It can be adjusted according to the training results.

[0207] After pre-training is complete, the model enters the fine-tuning phase. The goal of this phase is to allow the model to adapt parameters on rare earth compound data to learn its special energy formation mechanism and feasibility rules. The fine-tuning phase uses the same three-layer GAT architecture as pre-training, but only updates the parameters of the last layer GAT and the two fully connected prediction heads, the first two layers are frozen to prevent catastrophic forgetting.

[0208] The fine-tuning phase only updates the parameter set of the last layer GAT and the two prediction heads, that is:

[0209]

[0210] The optimization goal is defined as:

[0211]

[0212] The Adam optimizer is used, with a learning rate of , a batch size of 16, and early stopping (stop if the loss does not rise for 10 consecutive rounds) is used during training to prevent overfitting.

[0213] Part IV Prediction Results

[0214] After the model fine-tuning is complete, the second part of the embedding is completed. The input to the trained GAT model is the measured rare earth compound to be tested. For each compound graph, the model outputs two results: formation energy prediction value and feasibility prediction probability.

[0215] ​​The input graph is processed by three layers of GAT to extract the graph-level vector representation. Then, two parallel fully connected layers are used to predict the formation energy and the feasibility, respectively.

[0216]

[0217]

[0218] To facilitate sorting and screening, the formation energy and the feasibility are combined into a comprehensive score:

[0219]

[0220] where a compound with lower formation energy (more stable) gets a higher score, to prevent extremely small values due to division by zero.

[0221] According to the comprehensive score to determine whether the compound can exist stably. Finally, after sorting all the compounds, the compounds with higher scores are considered as potential feasible rare earth compounds for experimental verification.

[0222] An example is given to illustrate the prediction process:

[0223] Taking the unknown rare earth compound NdSe2 as an example, the complete prediction process of the method of the present application is described below.

[0224] First, in the element combination stage, the rare earth element set and the common anion element set are arranged and combined to obtain a plurality of candidate binary compounds. Then, according to the chemical rationality rules, such as excluding noble gases, maintaining valence balance, and avoiding meaningless matching, the candidate set that may exist is obtained. Assuming that the Nd–Se combination passes the screening and NdSe2 does not appear in the public compound database, NdSe2 is input into the prediction model as an unknown sample to be tested.

[0225] Next, according to the graph representation defined in the pre-trained model, the compound is constructed into an undirected graph. Each atom is regarded as a node, and the node features include atomic number, electronegativity, atomic radius, and rare earth flag. The edges between nodes are established according to the element co-occurrence relationship, and the edge features consist of the difference in electronegativity and the difference in atomic radius. The rare earth element nodes are amplified by weighting before input to highlight the influence of rare earth elements on the overall properties. The weighting process can be represented as:

[0226]

[0227] where, is the original node feature, is the rare earth node enhancement coefficient.

[0228] ​The constructed graph is input into a pre-trained graph attention network (GAT) fine-tuned for rare-earth data. The model obtains the overall compound embedding vector through multi-layer attention aggregation. Subsequently, two parallel fully connected branches output the compound formation energy. (Regression) and Feasibility Probability (Classification). The model output format is as follows:

[0229]

[0230] in, The diagram shows the structure of the compound.

[0231] Ultimately, a comprehensive evaluation index will be obtained by weighting and combining the feasibility probability with the overall feasibility probability. This is used for sorting and filtering. The comprehensive scoring formula is:

[0232]

[0233] Score The higher the score, the more energy-stable the compound is, and the greater the likelihood that the model believes it can form. After calculating the scores for all candidate samples, the high-scoring samples can be selected based on this ranking.

Claims

1. A rare earth compound prediction method based on pre-training fine-tuning of a graph neural network, characterized in that, Includes the following steps: Step S1: Generate a set of candidate rare earth compounds: Generate binary compound combinations of rare earth elements and other elements by permutation and combination, and screen them according to chemical rules and known compound databases to obtain a set of candidate rare earth compounds. Step S2: Compound graph embedding: Each compound in the candidate rare earth compound set is converted into graph structure data, which includes node features and edge features, wherein the node features contain rare earth flag variables for identifying rare earth elements; Step S3: Compound property prediction: Input the graph structure data into a pre-trained and fine-tuned graph neural network model, and the model outputs the predicted formation energy and predicted feasibility probability of each candidate rare earth compound. Step S4: Compound screening: Calculate the comprehensive score of each candidate rare earth compound based on the predicted formation energy and predicted feasibility probability, and sort and screen the candidate rare earth compounds according to the comprehensive score.

2. The method of claim 1, wherein, The screening process in step S1, based on chemical rules, includes: Excluding compounds of rare earth elements and inert gases; and / or Combinations of compounds that do not satisfy the principle of valence balance are excluded, where the principle of valence balance means that the sum of the oxidation states of the elements in the compound is zero.

3. The method of claim 2, wherein: Step S1 includes: Step S11: Generate compound combinations: By permutation and combination, combine the rare earth element set with elements from other element sets in pairs to generate all possible binary compound combinations, where: The rare earth element set is {La, Ce, Nd, Sm, Eu, Gd, Tb, Dy, Ho, Er, Tm, Yb, Lu}; The set of other elements is {O, N, S, C, Al, Fe, Ca, Ba}; The compound combination is generated by the following formula: in, and These represent indices for rare earth elements and other elements, respectively. Step S12: Rule-based screening: Screening the compound combinations to exclude combinations that do not conform to scientific principles, specifically including: Excluding combinations of rare earth elements and inert gases, i.e.: in, and These represent the indices for rare earth elements and inert gases, respectively. Exclude combinations that do not satisfy the principle of valence equilibrium, i.e.: in, and Representing elements respectively and elements The number of charges exhibited in a compound; Step S13: Screening the chemical database: For the remaining compound combinations after rule-based screening, exclude compounds that already exist in the chemical database by querying it. Here, f(compound combination) refers to whether the compound already exists in the database when the API query returns the result.

4. The method according to claim 1, characterized in that: Step S2 includes: Step S21: Constructing a graph structure: Representing the candidate rare earth compounds as an undirected graph ,in Let be a set of nodes, representing atoms in a compound. The set of edges represents the bonding relationships between atoms. The construction of these edges is based on the principle of valence state and chemical valence balance, and edges are only established between rare earth elements and non-rare earth elements. Step S22: Construct node features: for each node Constructing node feature vectors ,in is the atomic number of the element. The electronegativity of an element Where is the atomic radius of the element. This is a variable indicating the rare earth element; when the element is a rare earth element... ,otherwise ; Step S23: Construct edge features: For each edge Constructing edge feature vectors ,in, For nodes With nodes The absolute value of the electronegativity difference, For nodes With nodes The absolute value of the difference in atomic radii; Step S24: Rare Earth Node Weighting: Weighting of Rare Earth Element Flag Variables For each node, its feature vector is weighted and amplified to obtain the updated node feature vector. ,in Rare earth node enhancement coefficients greater than 1 , For the number of valence electrons, Coordination number; Step S25: Generate graph representation: Construct node feature matrix based on the updated feature vectors of all nodes. And construct an adjacency matrix based on the edge relationships between nodes. If there exists an edge but 1, otherwise Finally, the graphical structure of the candidate rare earth compounds was obtained. .

5. The method according to claim 1, characterized in that, In step S3, the training method for the pre-trained and fine-tuned graph neural network model includes: Pre-training phase: Using a dataset of common compounds that do not contain rare earth elements, train a graph neural network model to learn how to predict the formation energy and feasibility of compounds; Fine-tuning stage: Using known rare earth compound datasets, the parameters of the pre-trained model are fine-tuned to adapt the model to the characteristics of rare earth compounds.

6. The method according to claim 5, characterized in that: Step S3 includes: Step S31: Model Construction: Construct a dual-output structure model based on graph attention network (GAT). The model takes the graph structure data of the compound as input, extracts features through multiple layers of GAT, and predicts the formation energy and feasibility probability of the compound through two parallel output branches. Step S32: Model pre-training: The model is pre-trained using a dataset of common compounds that do not contain rare earth elements. The goal of the pre-training is to simultaneously optimize the formation of regression tasks and feasibility classification tasks. Step S33: Model fine-tuning: Using the known rare earth compound dataset, fine-tuning is performed on the pre-trained model. The fine-tuning process freezes the parameters of the graph neural network layers except for the last layer, and only updates the parameters of the last graph neural network layer and its two output branches. Step S34: Joint Loss Function Optimization: A weighted joint loss function is used in both the pre-training and fine-tuning stages. The model is optimized, and the joint loss function is defined as follows: in, and These are hyperparameters used to balance the importance of regression and classification tasks; To form the mean squared error loss for a regression task, it is defined as: Where B is the batch size. Let i be the predicted formation energy of the i-th sample. Let Lfeas be the true formation energy of the i-th sample; Lfeas is the binary cross-entropy loss of the feasibility classification task, defined as: in, Let {0 or 1} be the true feasibility label for the i-th sample. Let be the probability of feasibility of the i-th sample predicted by the model; Step S35: Prediction Execution: Input the graphical structure data of the rare earth compound to be tested into the pre-trained and fine-tuned model to obtain the predicted formation energy and predicted feasibility probability of the compound.

7. The method according to claim 6, characterized in that: A three-layer graph attention network (GAT) is used for node feature encoding. In the graph neural network encoding stage, the main task of each GAT layer is to fuse the local neighbor information of nodes to generate node embeddings, enabling the node features to reflect the structure and elemental properties of the compound graph. The processing flow of each layer is as follows: Linear mapping: For each node i in layer l and the k-th attention head, the node features are transformed using a linear transformation. Mapping to a low-dimensional space yields the mapped node features. And for edge features Perform a linear mapping to obtain the mapped edge features. ; The mapping process is as follows: in, and The weight matrix is ​​a learnable matrix. Let l be the dimension of the input features of the l-th layer. For the dimension of attention head, Let be the dimension of the edge features. Indicates transpose; Calculate the attention score: For node i and its neighbor node j, calculate the attention score. This score takes into account both the mapped node features and edge features; The calculation process is as follows: in, For learnable vectors, For activation function, This represents a vector concatenation operation; Attention Normalization and Dropout: Normalize the attention scores of all neighbors of node i, and apply Dropout to obtain the normalized attention weights. The normalization and Dropout process is as follows: in, The probability of Dropout; Node aggregation and update: Based on the normalized attention weights, the mapping features of neighboring nodes are weighted and summed, and then processed through a non-linear activation function. Update node features to obtain the output of the k-th attention head. The update process is as follows: Let ii be the set of neighboring nodes; Multi-head merging and residual connections: For GATs that are not in the last layer, the outputs of all attention heads are concatenated and combined with residual connections and layer normalization to obtain the input of the next layer. The merging process is as follows: Where K is the number of attention heads, and LayerNorm is the layer normalization operation. For residual projection operations; Final layer aggregation: For the last GAT layer, the outputs of all attention heads are averaged and combined with residual connections and layer normalization to obtain the final node embedding. The final polymerization process is as follows: in, The number of attention heads in the last layer.

8. The method according to claim 6, characterized in that: After the graph attention network is encoded, the node embeddings are converted into graph-level features, and dual-task prediction is performed through a fully connected layer. The specific steps include: a) Graph-level feature pooling: Pooling is performed on the node embeddings obtained after encoding by the graph neural network to generate a graph-level feature vector. The pooling method is average pooling, and the calculation formula is: Where N is the total number of nodes in the graph. This represents the embedding representation of the i-th node in the last layer. b) Forming regression prediction: The graph-level feature vectors The input is fed into a formation energy regression branch, which predicts the formation energy of the compound through a fully connected layer. The prediction process is as follows: in, and These are the weight matrix and bias term that form the regression branch, respectively; c) Feasibility classification prediction: The graph-level feature vectors... The inputs are fed in parallel to a feasibility classification branch, and the feasibility probability of the compound is predicted through a fully connected layer. The prediction process is as follows: in, , These are the weight matrix and bias term for the feasibility classification branch, respectively.

9. The method according to claim 6, characterized in that: Step s33, model fine-tuning, includes the following steps: Step s331: Freeze the parameters of the graph neural network layers in the pre-trained model except for the last layer, and only update the parameters of the last layer graph attention network (GAT) and the two fully connected prediction heads to prevent catastrophic forgetting; Step s332: Define the set of parameters to be updated as follows Where GAT(3) are the parameters of the last layer of GAT. and To form weights and biases that can predict branches, and Weights and biases for the feasibility prediction branch; Step s333: Solve for the optimal parameter set with minimizing the loss function LL as the optimization objective. The optimization objective is defined as: in, Let L be the optimal parameter set and L be the loss function; Step s334: Optimize using the Adam optimizer with a learning rate of 1×10⁻⁶. −4 The batch size is 16, and an early stop strategy is used during training. If the loss does not decrease for 10 consecutive rounds, training is stopped.

10. The method according to claim 1, characterized in that: Step S4 includes: Obtain the predicted formation energy and predicted feasibility probability of the rare earth compound to be tested; Based on the predicted formation energy and predicted feasibility probability, a comprehensive score is calculated for each rare earth compound to be tested. : in, The overall score for the i-th compound is... Let i be the predicted formation energy of the i-th compound. Let be the predicted feasibility probability of the i-th compound. This represents the maximum predicted formation energy among all the rare earth compounds tested. This represents the minimum predicted formation energy among all the rare earth compounds tested. To prevent the minimum value of division by zero; Based on the comprehensive score All rare earth compounds to be tested were sorted, and compounds with higher comprehensive scores were selected as potentially feasible rare earth compounds.

Images