A surface relaxation catalyst adsorption energy prediction method based on a graph neural network
By introducing long-distance skip connections and residual attention mechanisms into graph neural networks, combined with multi-objective loss functions, the accuracy and reliability of electrochemical catalyst adsorption energy prediction are improved, solving the problems of high computational cost and low prediction accuracy in existing technologies, and achieving more stable physical relationship capture and noise processing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- DALIAN MARITIME UNIVERSITY
- Filing Date
- 2026-04-07
- Publication Date
- 2026-07-21
Smart Images
Figure CN122436038A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of catalyst adsorption energy prediction technology, specifically to a method for predicting the adsorption energy of surface relaxor catalysts based on graph neural networks. Background Technology
[0002] Predicting the catalytic performance of electrochemical reaction catalysts is an important research direction in materials science, with the core being the accurate assessment of the catalyst's adsorption capacity for reaction intermediates. Catalyst performance is typically measured by key parameters such as adsorption energy, which are closely related to the catalyst's electronic structure. Traditional electronic structure analysis methods, such as density functional theory (DFT) calculations, can provide precise atomic-level information, but their high computational complexity makes them difficult to apply to large-scale catalyst screening. In recent years, machine learning methods have shown potential in materials property prediction, especially graph neural networks (GNNs), which have attracted widespread attention due to their ability to directly process graph structure data of molecules or crystals. Graph neural networks automatically learn the feature representations of atoms and their chemical environments through message passing mechanisms, avoiding the tedious feature engineering of traditional machine learning methods and providing a new technical approach for efficiently predicting catalyst performance.
[0003] Currently, the prediction of electrochemical catalyst performance mainly relies on first-principles calculations and traditional machine learning methods. While first-principles calculations offer high accuracy, their high computational resource consumption and long processing time limit their application in high-throughput screening. Traditional machine learning methods (such as support vector machines and random forests), despite their high computational efficiency, depend on manually designed features and struggle to fully capture the complex interrelationships of catalyst structures, thus limiting prediction accuracy. Furthermore, existing methods often require additional correction steps when dealing with surface relaxation effects, further increasing the computational burden. Although graph neural networks have made some progress in predicting material properties, they still face challenges in predicting catalyst adsorption energies, especially for accurately modeling adsorption energies under surface relaxation conditions. Existing technologies have not yet fully addressed how to efficiently integrate structural relaxation information with graph neural network models, affecting the reliability and universality of prediction results. Summary of the Invention
[0004] To address the aforementioned technical problems of high computational cost and long processing time associated with traditional computational methods, and the difficulty in capturing the structural-property relationships of chemical processes due to extensive feature engineering and data preprocessing required by traditional machine learning methods, this invention provides a method for predicting the adsorption energy of surface relaxation catalysts based on graph neural networks. This invention effectively improves the reliability of the model's learning of deep information by introducing long-distance skip connections and residual attention mechanisms into the architecture; and effectively enhances the model's ability to adjust noise using real physicochemical properties through multi-objective loss design.
[0005] The technical means employed in this invention are as follows:
[0006] A method for predicting the adsorption energy of surface relaxor catalysts based on graph neural networks includes the following steps: Acquire catalyst crystal structure data and adsorbate structure data, construct a catalyst-adsorbate molecular map based on the catalyst crystal structure data and adsorbate structure data, and construct a dataset based on the molecular map; The dataset is divided into a training set, a validation set, and a test set; An adsorption energy prediction model is constructed based on an equivariant graph neural network. The adsorption energy prediction model includes a feature allocation layer, an embedding layer, a Transformer module, a scalar gated jump fusion module, and a third normalization layer connected in sequence. The Transformer module includes a first normalization layer, a graph residual attention layer, a second normalization layer, and a forward propagation network connected in sequence. The scalar gated jump fusion module uses the features of the initial layer, the intermediate layer, and the final layer to perform feature fusion. The adsorption energy prediction model is trained using a training set and a validation set. During the training process, a multi-objective loss function is constructed, which includes mean absolute error, mean square error, and Huber loss. The parameters of the adsorption energy prediction model are updated using the multi-objective loss function. The test set is input into the trained adsorption energy prediction model, and the adsorption energy prediction result is output.
[0007] Furthermore, the Transformer module includes n sequentially connected Transformer layers, and the workflow of the adsorption energy prediction model includes: The input molecular map is fed into the feature allocation layer to obtain the input features of the crystal map; The input features of the crystal diagram are input into the embedding layer to obtain the initial node features after scale normalization; The scale-normalized initial features of the nodes are input into the Transformer module to obtain the output of the nth Transformer layer and the output of the second nth Transformer layer. The input features of the crystal diagram and the output of the Transformer layer of the second nth layer are fused by mean averaging. The features after mean fusion and the output of the nth Transformer layer are passed through the scalar gated jump fusion module to obtain the output of the scalar gated jump fusion module; The output of the scalar gated jump fusion module is passed through the third normalization layer to obtain the output of the adsorption energy prediction model.
[0008] Furthermore, the workflow of the Transformer layer includes: The input features of the Transformer layer are fed into the first normalization layer to obtain the node feature input of the graph residual attention layer; The node features of the graph residual attention layer are input into the graph residual attention layer to obtain the output of the graph residual attention layer; The output of the graph residual attention layer and the input features of the Transformer layer are added together to obtain the summed features. The summed features are input into the second normalization layer to obtain the output of the second normalization layer; The output of the second normalization layer is input into the forward propagation layer to obtain the output of the forward propagation layer; The summed features are added to the output of the forward propagation layer to obtain the output features of the Transformer layer.
[0009] Furthermore, the workflow of the feature allocation layer includes: Each atom in the input molecular graph is treated as a node, and the neighborhood pairs of atoms and their neighbors are treated as edges. For a node, the element type, the three-dimensional coordinates of each atom, and the atom category are assigned as input node features; For edges, the connection relationships of the edges, periodic cell offsets, and element type information are assigned to calculate the relative displacements and distances between atoms; By expanding the interatomic distance using radial basis functions and combining the rotation matrix and edge direction information, a local coordinate system and rotation operator defined by the edge direction are generated to obtain the input features of the crystal diagram. By using a cutoff radius to limit the range of the atomic neighborhood, the model learns the local interaction rules within the atomic neighborhood.
[0010] Furthermore, the workflow of the embedding layer includes: After encoding the atom types of the input features of the crystal diagram, the initial node feature components are obtained; The initial node feature components are mapped to edge-locally equivariant features with an irreps structure using an equivariant linear mapping. The local isomorphic features of the edges are aligned to the global coordinate system using a reverse rotation matrix; Multiply the local isovariant features of the aligned edges with the distance features to obtain the edge messages of the neighbors; The neighbor edge messages are aggregated into the node initial features, and the local geometric structure and connectivity information are encoded into the aggregated node initial features. The encoded node initial features are then scaled and normalized.
[0011] Furthermore, the workflow of the first normalization layer includes: The input features of the Transformer layer are divided into a first scalar part and a first higher-order part; The first scalar portion is normalized using standard LayerNorm, calculated as follows:
[0012] in, The first scalar part after normalization. For the first channel-wise learnable scaling parameter, For element-wise multiplication, This refers to the 0th-order scalar part of the first scalar part. The mean of a zero-order vector. The standard deviation of a zero-order vector. The mean of the 0th-order vector is calculated using the formula for the second channel-wise learnable bias parameter:
[0013] in, The number of feature channels, For the 0th order, the 0th order, and the 0th order in the first scalar part i The scalar coefficients corresponding to each channel, and the formula for calculating the standard deviation of the 0th-order vector is: ; The first higher-order part is normalized using RMS, and the calculation formula is as follows:
[0014] in, This is the first higher-order part after normalization. For the third channel-wise learnable scaling parameter, The first higher-order part L Irreducible representation of feature blocks of order 1 The overall root mean square value is calculated using the following formula:
[0015] in, For the largest order, Let be the root mean square value of the Lth-order feature. Let L be the order of the feature. The root mean square value of the Lth-order feature is calculated using the following formula:
[0016] in, For the first higher-order part L order, m-th order, and order m i scalar coefficients corresponding to each channel; Based on the normalized first scalar part and the normalized first higher-order part, the node feature input of the graph residual attention layer is obtained.
[0017] Furthermore, the workflow of the graph residual attention layer includes: Based on the relative position vectors obtained from the atoms, the rotation matrix is used to rotate and align the node feature inputs of the graph residual attention layer. Distance information is embedded into the rotated and aligned node features using radial functions and radial basis functions, and edge-level intermediate features are obtained through isovariant linear operations. The edge-level intermediate feature is decomposed into a second scalar part and a second higher-order part; In the second scalar part, LayerNorm is used to normalize the second scalar part, and the normalized second scalar part is used to obtain the logits value by applying the LeakyReLU activation function and linear mapping. In the neighborhood dimension, Softmax is used to normalize the logits value to obtain the original attention weights. The edge feature prediction gate is fused with the logits value to generate the edge-modulated gating attention weights; Linear interpolation is performed between the original attention weights and the edge-modulated gated attention weights to obtain the residual stabilized attention weights; The second higher-order part is projected onto the spherical sampling points to form a spherical sampling domain. A nonlinear transformation is performed on the spherical sampling domain, and the nonlinearly transformed spherical sampling domain is projected onto the irreps space to obtain the activated isovariant features. By using linear operations to perform learnable isovariant linear mixing on the activated isovariant features, we can obtain the isovariant features of the edges. The output of the graph residual attention layer is obtained by fusing the residual stabilization attention weights and the equivariant features of the edges.
[0018] Furthermore, the workflow of the scalar gated jump fusion module includes: Based on the scale-normalized initial node features, the output of the second (n)th layer of the Transformer layer, and the output of the nth layer of the Transformer layer, dynamic weights are calculated. The formula for calculating the dynamic weights is as follows:
[0019] in, For dynamic weights, For learnable scalars, For dynamic increments; For the scalar components of the features after mean fusion and the final layer features, the enhanced node features are calculated using gating and the dynamic weights. The formula for calculating the enhanced node features is as follows:
[0020] in, For the enhanced node features, These are scalar features among the features obtained after mean fusion. Scalar features for the final layer features For gating, For element-wise multiplication; For the equivariant components of the features after mean fusion and the final layer features, the enhanced equivariant features are calculated using dynamic weights. The formula for calculating the enhanced equivariant features is as follows:
[0021] in, For the enhanced isovariant features, The equivariant features of the final layer features, These are the equivariant features among the features after mean fusion; The enhanced node features and the enhanced isotropic features are added together to obtain the output of the scalar gated jump fusion module.
[0022] Furthermore, the calculation formula for the multi-objective loss function is as follows:
[0023] in, For multi-objective loss functions, The learnable log-variance parameter of the mean absolute error loss. Let the mean absolute error loss function be . The learnable log-variance parameter for the mean squared error loss. Let the mean squared error loss function be . The learnable log-variance parameter of the Huber loss. Here is the Huber loss function.
[0024] Compared with the prior art, the present invention has the following advantages: 1. This invention applies the residual stabilization mechanism to the attention layer and proposes a key component called the graph residual attention layer. It introduces learnable attention weights to enhance the residuals and establishes a gradient stability framework for the attention layer through a gating mechanism.
[0025] 2. The invention introduces lightweight long-distance jump connections at the architecture level to solve the problems of large number of parameters and severe information attenuation in multi-layer fusion. It uses three points—initial layer features, intermediate layer features, and final layer features—to locate the information flow path and uses dynamic weights and scalar gating parameters to achieve a lightweight design for global control.
[0026] 3. To address the issue of varying sensitivities to different energies during the training phase, this invention proposes a physicochemical-based multi-objective loss mechanism. It utilizes three types of losses—MAE, MSE, and Huber—to achieve multi-scale capture, effectively handling noise and outliers in the training data. Physicochemical properties are incorporated into the loss function design, and the resulting total loss is then backpropagated to update the model parameters, thereby improving the accuracy and generalization ability of adsorption energy prediction.
[0027] 4. This invention first utilizes a residual attention mechanism to enhance the stability of the attention level, enabling the model to capture real physical relationships more stably; secondly, in terms of architecture, it utilizes long-distance skip connections to enable the model to obtain stronger physical laws in local and long-range structures; thirdly, the design of a multi-objective loss function allows the model to adjust noise using more realistic physical and chemical properties during training.
[0028] Based on the above reasons, this invention can be widely applied in fields such as deep learning, first-principles calculations, hydrogen evolution reaction, and catalyst adsorption energy prediction. Attached Figure Description
[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0030] Figure 1 This is a flowchart of the catalyst adsorption energy prediction method of the present invention.
[0031] Figure 2 This is a model diagram of the isovariant graph neural network based on residual stabilization attention in this invention.
[0032] Figure 3 This is a model diagram of the lightweight long-distance jump connection mechanism of the present invention. Detailed Implementation
[0033] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0034] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0035] like Figure 1 As shown, this invention provides a method for predicting the adsorption energy of surface relaxor catalysts based on graph neural networks, comprising the following steps: S1. Obtain catalyst crystal structure data and adsorbate structure data, construct a catalyst-adsorbate molecular map based on the catalyst crystal structure data and adsorbate structure data, and construct a dataset based on the molecular map.
[0036] Specifically, data screening and crystal format conversion were performed on typical adsorbate-based adsorption substrate crystal structure data collected and organized. First, the adsorbates were screened, retaining only materials with H adsorbates. Next, data was screened based on the elemental composition of the crystals, removing crystal structures containing non-metallic elements other than the adsorbate H, ensuring that all substrate alloys in the dataset consisted of only two metallic elements. Then, the format of the collected JSON data was converted, and basic atomic information of the crystal structure was extracted. Unique identifiers for material samples, neighbor lists, and constraint atom indices were constructed, and the relaxation energy and initial energy of the crystals were extracted. Finally, a pyG Data object was constructed based on the above information, adapted to the standard input format of the model, and written to lmdb, completing the format conversion.
[0037] S2. Divide the dataset into training set, validation set and test set.
[0038] S3. Construct an adsorption energy prediction model. The adsorption energy prediction model is based on an equivariant graph neural network. The adsorption energy prediction model includes a feature allocation layer, an embedding layer, a Transformer module, a scalar gated jump fusion module, and a third normalization layer connected in sequence. The Transformer module includes a first normalization layer, a graph residual attention layer, a second normalization layer, and a forward propagation network connected in sequence. The scalar gated jump fusion module uses the features of the initial layer, the intermediate layer, and the final layer to perform feature fusion.
[0039] The Transformer module consists of n sequentially connected Transformer layers. Each Transformer layer includes a first normalization layer, a graph residual attention layer, a second normalization layer, and a forward propagation network, all connected in sequence.
[0040] like Figure 2 As shown, the dashed box contains the specific network architecture of a Transformer layer.
[0041] The workflow of the adsorption energy prediction model includes: The first step is to input the molecular map into the feature allocation layer to obtain the input features of the crystal map.
[0042] The second step is to input the crystal map's input features into the embedding layer to obtain the scale-normalized initial node features.
[0043] The third step is to input the scale-normalized initial node features into the Transformer module to obtain the output of the nth Transformer layer and the output of the second nth Transformer layer.
[0044] The fourth step is to perform mean fusion of the input features of the crystal diagram and the output of the Transformer layer of the second nth layer.
[0045] Step 5: Combine the features after mean fusion and the output of the nth Transformer layer with the scalar gated jump fusion module to obtain the output of the scalar gated jump fusion module.
[0046] Step 6: Pass the output of the scalar gated jump fusion module through the third normalization layer to obtain the output of the adsorption energy prediction model.
[0047] The workflow of the Transformer layer includes: The first step is to input the input features of the Transformer layer into the first normalization layer to obtain the node feature input of the graph residual attention layer.
[0048] The second step is to input the node features of the graph residual attention layer into the graph residual attention layer to obtain the output of the graph residual attention layer.
[0049] The third step is to add the output of the graph residual attention layer and the input features of the Transformer layer to obtain the summed features.
[0050] The fourth step is to input the summed features into the second normalization layer to obtain the output of the second normalization layer.
[0051] Step 5: Input the output of the second normalization layer into the forward propagation layer to obtain the output of the forward propagation layer.
[0052] Step 6: Add the summed features to the output of the forward propagation layer to obtain the output features of the Transformer layer.
[0053] The workflow of the feature allocation layer includes: The first step is to treat each atom of the input molecular graph as a node. The neighborhood pairs (i,j) formed by atom i and its neighbor j are used as edges.
[0054] The second step is to assign element type, three-dimensional coordinates of each atom, and atom category as input node features for each node.
[0055] The third step involves assigning edge connection relationships, periodic cell offsets, and element type information to the edges to calculate the relative displacements and distances between atoms.
[0056] The fourth step involves expanding the interatomic distances using radial basis functions, combining the rotation matrix and edge direction information to generate a local coordinate system and rotation operator defined by the edge directions, thus obtaining the input features of the crystal diagram.
[0057] The fifth step is to use a cutoff radius to limit the range of the atomic neighborhood, so that the model can learn the local interaction rules within the atomic neighborhood.
[0058] The workflow of the embedding layer includes: The first step is to encode the atom types of the input features of the crystal diagram (mapped to a uniform dimension) to obtain the initial node feature components.
[0059] The second step is to use SO(2) equivariant linear mapping to map the initial node feature components into edge local equivariant features with irreps structure.
[0060] Step 3: Using the reverse rotation matrix Align the local equivariant features of the edges to the global coordinate system to ensure the equivariance of SE(3) during aggregation.
[0061] The fourth step is to multiply the local isovariant features of the aligned edges with the distance features to obtain the edge messages of the neighbors, so that the strength of the inter-atomic interaction is adaptively adjusted with the inter-atomic distance.
[0062] The fifth step is to aggregate the edge messages of neighbors into the initial features of the node, encode the local geometric structure and connectivity information into the aggregated initial features of the node, and perform scale normalization on the encoded initial features of the node to maintain numerical stability.
[0063] The workflow of the first normalization layer includes: The first step is to divide the input features of the Transformer layer into a first scalar part and a first higher-order part.
[0064] The second step is to perform standard LayerNorm normalization on the first scalar part, calculated using the following formula:
[0065] in, The first scalar part after normalization. For the first channel-wise learnable scaling parameter, For element-wise multiplication, This refers to the 0th-order scalar part of the first scalar part. The mean of a zero-order vector. The standard deviation of a zero-order vector. The formula for calculating the mean of the 0th-order vector, which is the second channel-wise learnable bias parameter, is as follows:
[0066] in, The number of feature channels, For the 0th order, the 0th order, and the 0th order in the first scalar part i The formula for calculating the standard deviation of the scalar coefficients corresponding to each channel and the 0th-order vector is: ; The third step is to apply RMS normalization to the first higher-order component without reducing the mean, preserving the relative amplitude relationship between higher-order components, and stabilizing the scale of different components. The calculation formula is as follows:
[0067] in, This is the first higher-order part after normalization. For the third channel-wise learnable scaling parameter, The first higher-order part L Irreducible representation of feature blocks of order 1 This is the global root mean square value, and the formula for calculating the global root mean square value is:
[0068] in, For the largest order, Let be the root mean square value of the Lth-order feature. Let L be the order of the feature. The root mean square value of the Lth-order feature is calculated using the following formula:
[0069] in, For the first higher-order part Lorder, m-th order, and order m i The scalar coefficients corresponding to each channel.
[0070] The fourth step is to obtain the node feature input of the graph residual attention layer based on the normalized first scalar part and the normalized first higher-order part.
[0071] The workflow of the graph residual attention layer includes: The first step is to obtain the relative position vectors based on the atoms and then use the rotation matrix to rotate and align the node feature inputs.
[0072] Specifically, splicing and The node features, based on the relative position vectors of atoms. Using rotation matrix Rotate and align it.
[0073] The second step is to use radial functions and radial basis functions to obtain distance information. The node features embedded in the rotation-aligned nodes are then subjected to SO(2) isovariant linear operations to obtain the edge-level intermediate features. .
[0074] Step 3: Transfer the intermediate features at the edge level Disassemble into the second scalar part Second higher order part .
[0075] Step 4: In the second scalar part, use LayerNorm to process the second scalar part. Normalization is performed to stabilize the scalar feature distribution of different edges. The logits value is then obtained by applying the LeakyReLU activation function and linear mapping to the normalized second scalar part. The original attention weights are obtained by normalizing the logits values using Softmax in the neighborhood dimension. The formula is as follows:
[0076] in, For the original attention weights, For nodes i with neighboring nodes j The corresponding attention logits value on the edge, For nodes i with neighboring nodes k The attention logits value on the corresponding edge.
[0077] Step 5: Predict gating using edge features Combined with logits values, edge-modulated gating attention weights are generated. The formula is as follows:
[0078]
[0079] in, Let be the scalar characteristic of the edge. It is the Sigmoid activation function. Here is the mapping function for the multilayer perceptron used for distance-gated prediction. Gating for edge feature prediction Modulate the gating attention weights for the edge.
[0080] Step 6: Adjust the original attention weights And edge modulation gating attention weights Linear interpolation is performed to obtain the residual stabilization attention weights. :
[0081] in, To stabilize the attention weights of the residuals, Here, the weight is defined by the formula:
[0082] in, For learnable residual gating parameters, For learnable residual attention weight parameters, For the truncation function, The interval is restricted to [0,1].
[0083] Step 7: Utilize separability The activation process projects the second higher-order part onto the spherical sampling points to form a spherical sampling domain. A nonlinear transformation is performed on the spherical sampling domain to maintain the rotationally equivariant structure. The nonlinearly transformed spherical sampling domain is then projected onto the Irreps space to obtain the activated equivariant features.
[0084] Step 8: Use the SO(2) linear operation to perform a learnable isovariant linear mixture on the activated isovariant features to obtain the isovariant features of the edges. .
[0085] Step 9: After fusing the residual stabilization attention weights and the equivariant features of the edges, the output of the graph residual attention layer is obtained. .
[0086] After the attention is updated, it passes through a separable normalization layer again, and then is input into the forward propagation network using two linear layers as external projections and separable normalization layers. Activate nonlinear feature transformation and channel fusion.
[0087] Long-distance jump connection mechanism such as Figure 3 As shown: Step 1: Select initial layer features (initial node features after scale normalization) Intermediate layer features (output of the Transformer layer at the second nth layer) Final layer features (output of the nth Transformer layer) The fusion process first utilizes mean fusion to aggregate the initial layer features. Features of the intermediate layer The formula is as follows:
[0088] in, These are the features after mean fusion.
[0089] The second step involves using a scalar-gated skip fusion module (ISGS) to inject the mean-aggregated features into the final representation via residual injection. This utilizes learnable scalars. Solve the problem of fixed global coefficients, and based on , , Three-segment feature adaptive prediction node-level dynamic increment Output dynamic weights The formula for calculating dynamic weights is:
[0090] in, For dynamic weights, For learnable scalars, It is a dynamic increment.
[0091] The third step involves further analyzing the scalar components of the features obtained after mean fusion and the final layer features using gating. To improve training stability, the formula is as follows:
[0092] Fourth step: For the scalar part, learn the dynamic weights. At the same time, using gating It controls the injection of long-range information into different channels and outputs enhanced node features. The formula is as follows:
[0093] in, For the enhanced node features, These are scalar features among the features obtained after mean fusion. Scalar features for the final layer features For gating, This is for element-wise multiplication.
[0094] Fifth step: For the equivariant part, in order to maintain the equivariance of SO(3), only dynamic weights are added. Residual injection is performed on the skip branches to obtain enhanced isovariant features. The formula is as follows:
[0095] in, For the enhanced isovariant features, The equivariant features of the final layer features, These are the equivariant features among the features after mean fusion.
[0096] Step 6: After adding the enhanced node features and the enhanced isotropic features, the output of the scalar-gated skip fusion module is obtained. The long-distance skip connection mechanism obtains stronger physical laws while preserving the geometric details of the shallow and middle layers.
[0097] Specifically, addition refers to reassembling the two according to their corresponding orders.
[0098] S4. Train the adsorption energy prediction model using the training and validation sets. During the training process, construct a multi-objective loss function, which includes mean absolute error, mean square error, and Huber loss. Update the parameters of the adsorption energy prediction model using the multi-objective loss function.
[0099] A physicochemical-based multi-objective loss design is implemented as follows: Based on the model's predicted value y and the actual value y, three error metrics—MAE, MSE, and Huber—are obtained. MAE is the Mean Absolute Error Loss Function, MSE is the Mean Squared Error Loss Function, and Huber is a loss function that maintains smooth quadratic optimization in the small error range and transforms into a linear penalty in the large error range. Based on the different requirements of overall accuracy, large error penalty, and outlier noise in catalyst adsorption energy prediction, a learnable parameter sk is used to learn physicochemically meaningful weighting coefficients through homoscedastic uncertainty. The three types of losses are then weighted and fused to obtain the total loss. The calculation formula for the multi-objective loss function is as follows:
[0100] in, For multi-objective loss functions, The learnable log-variance parameter of the mean absolute error loss. Let the mean absolute error loss function be . The learnable log-variance parameter for the mean squared error loss. Let the mean squared error loss function be . The learnable log-variance parameter of the Huber loss. Here is the Huber loss function.
[0101] S5. Input the test set into the trained adsorption energy prediction model and output the adsorption energy prediction results.
[0102] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0103] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for predicting the adsorption energy of surface relaxation catalysts based on graph neural networks, characterized in that, Includes the following steps: Acquire catalyst crystal structure data and adsorbate structure data, construct a catalyst-adsorbate molecular map based on the catalyst crystal structure data and adsorbate structure data, and construct a dataset based on the molecular map; The dataset is divided into a training set, a validation set, and a test set; An adsorption energy prediction model is constructed based on an equivariant graph neural network. The adsorption energy prediction model includes a feature allocation layer, an embedding layer, a Transformer module, a scalar gated jump fusion module, and a third normalization layer connected in sequence. The Transformer module includes a first normalization layer, a graph residual attention layer, a second normalization layer, and a forward propagation network connected in sequence. The scalar gated jump fusion module uses the features of the initial layer, the intermediate layer, and the final layer to perform feature fusion. The adsorption energy prediction model is trained using a training set and a validation set. During the training process, a multi-objective loss function is constructed, which includes mean absolute error, mean square error, and Huber loss. The parameters of the adsorption energy prediction model are updated using the multi-objective loss function. The test set is input into the trained adsorption energy prediction model, and the adsorption energy prediction result is output.
2. The method for predicting the adsorption energy of surface relaxation catalysts based on graph neural networks according to claim 1, characterized in that, The Transformer module includes n sequentially connected Transformer layers, and the workflow of the adsorption energy prediction model includes: The input molecular map is fed into the feature allocation layer to obtain the input features of the crystal map; The input features of the crystal diagram are input into the embedding layer to obtain the initial node features after scale normalization; The scale-normalized initial features of the nodes are input into the Transformer module to obtain the output of the nth Transformer layer and the output of the second nth Transformer layer. The input features of the crystal diagram and the output of the Transformer layer of the second nth layer are fused by mean averaging. The features after mean fusion and the output of the nth Transformer layer are passed through the scalar gated jump fusion module to obtain the output of the scalar gated jump fusion module; The output of the scalar gated jump fusion module is passed through the third normalization layer to obtain the output of the adsorption energy prediction model.
3. The method for predicting the adsorption energy of surface relaxation catalysts based on graph neural networks according to claim 2, characterized in that, The workflow of the Transformer layer includes: The input features of the Transformer layer are fed into the first normalization layer to obtain the node feature input of the graph residual attention layer; The node features of the graph residual attention layer are input into the graph residual attention layer to obtain the output of the graph residual attention layer; The output of the graph residual attention layer and the input features of the Transformer layer are added together to obtain the summed features. The summed features are input into the second normalization layer to obtain the output of the second normalization layer; The output of the second normalization layer is input into the forward propagation layer to obtain the output of the forward propagation layer; The summed features are added to the output of the forward propagation layer to obtain the output features of the Transformer layer.
4. The method for predicting the adsorption energy of surface relaxation catalysts based on graph neural networks according to claim 2, characterized in that, The workflow of the feature allocation layer includes: Each atom in the input molecular graph is treated as a node, and the neighborhood pairs of atoms and their neighbors are treated as edges. For a node, the element type, the three-dimensional coordinates of each atom, and the atom category are assigned as input node features; For edges, the connection relationships of the edges, periodic cell offsets, and element type information are assigned to calculate the relative displacements and distances between atoms; By expanding the interatomic distance using radial basis functions and combining the rotation matrix and edge direction information, a local coordinate system and rotation operator defined by the edge direction are generated to obtain the input features of the crystal diagram. By using a cutoff radius to limit the range of the atomic neighborhood, the model learns the local interaction rules within the atomic neighborhood.
5. The method for predicting the adsorption energy of surface relaxation catalysts based on graph neural networks according to claim 2, characterized in that, The workflow of the embedding layer includes: After encoding the atom types of the input features of the crystal diagram, the initial node feature components are obtained; The initial node feature components are mapped to edge-locally equivariant features with an irreps structure using an equivariant linear mapping. The local isomorphic features of the edges are aligned to the global coordinate system using a reverse rotation matrix; Multiply the local isovariant features of the aligned edges with the distance features to obtain the edge messages of the neighbors; The neighbor edge messages are aggregated into the node initial features, and the local geometric structure and connectivity information are encoded into the aggregated node initial features. The encoded node initial features are then scaled and normalized.
6. The method for predicting the adsorption energy of surface relaxation catalysts based on graph neural networks according to claim 3, characterized in that, The workflow of the first normalization layer includes: The input features of the Transformer layer are divided into a first scalar part and a first higher-order part; The first scalar portion is normalized using standard LayerNorm, calculated as follows: in, The first scalar part after normalization. For the first channel-wise learnable scaling parameter, For element-wise multiplication, This refers to the 0th-order scalar part of the first scalar part. The mean of a zero-order vector. The standard deviation of a zero-order vector. The mean of the 0th-order vector is calculated using the formula for the second channel-wise learnable bias parameter: in, The number of feature channels, For the 0th order, the 0th order, and the 0th order in the first scalar part i The scalar coefficients corresponding to each channel, and the formula for calculating the standard deviation of the 0th-order vector is: ； The first higher-order part is normalized using RMS, and the calculation formula is as follows: in, This is the first higher-order part after normalization. For the third channel-wise learnable scaling parameter, The first higher-order part L Irreducible representation of feature blocks of order 1 The overall root mean square value is calculated using the following formula: in, For the largest order, Let be the root mean square value of the Lth-order feature. Let L be the order of the feature. The root mean square value of the Lth-order feature is calculated using the following formula: in, For the first higher-order part L order, m-th order, and order m i scalar coefficients corresponding to each channel; Based on the normalized first scalar part and the normalized first higher-order part, the node feature input of the graph residual attention layer is obtained.
7. The method for predicting the adsorption energy of surface relaxation catalysts based on graph neural networks according to claim 2, characterized in that, The workflow of the graph residual attention layer includes: Based on the relative position vectors obtained from the atoms, the rotation matrix is used to rotate and align the node feature inputs of the graph residual attention layer. Distance information is embedded into the rotated and aligned node features using radial functions and radial basis functions, and edge-level intermediate features are obtained through isovariant linear operations. The edge-level intermediate feature is decomposed into a second scalar part and a second higher-order part; In the second scalar part, LayerNorm is used to normalize the second scalar part, and the normalized second scalar part is used to obtain the logits value by applying the LeakyReLU activation function and linear mapping. In the neighborhood dimension, Softmax is used to normalize the logits value to obtain the original attention weights. The edge feature prediction gate is fused with the logits value to generate the edge-modulated gating attention weights; Linear interpolation is performed between the original attention weights and the edge-modulated gated attention weights to obtain the residual stabilized attention weights; The second higher-order part is projected onto the spherical sampling points to form a spherical sampling domain. A nonlinear transformation is performed on the spherical sampling domain, and the nonlinearly transformed spherical sampling domain is projected onto the irreps space to obtain the activated isovariant features. By using linear operations to perform learnable isovariant linear mixing on the activated isovariant features, we can obtain the isovariant features of the edges. The output of the graph residual attention layer is obtained by fusing the residual stabilization attention weights and the equivariant features of the edges.
8. The method for predicting the adsorption energy of surface relaxation catalysts based on graph neural networks according to claim 2, characterized in that, The workflow of the scalar gated jump fusion module includes: Based on the scale-normalized initial node features, the output of the second (n)th layer of the Transformer layer, and the output of the nth layer of the Transformer layer, dynamic weights are calculated. The formula for calculating the dynamic weights is as follows: in, For dynamic weights, For learnable scalars, For dynamic increments; For the scalar components of the features after mean fusion and the final layer features, the enhanced node features are calculated using gating and the dynamic weights. The formula for calculating the enhanced node features is as follows: in, For the enhanced node features, These are scalar features among the features obtained after mean fusion. Scalar features for the final layer features For gating, For element-wise multiplication; For the equivariant components of the features after mean fusion and the final layer features, the enhanced equivariant features are calculated using dynamic weights. The formula for calculating the enhanced equivariant features is as follows: in, For the enhanced isovariant features, The equivariant features of the final layer features, These are the equivariant features among the features after mean fusion; The enhanced node features and the enhanced isotropic features are added together to obtain the output of the scalar gated jump fusion module.
9. The method for predicting the adsorption energy of surface relaxation catalysts based on graph neural networks according to claim 1, characterized in that, The formula for calculating the multi-objective loss function is as follows: in, For multi-objective loss functions, The learnable log-variance parameter of the mean absolute error loss. Let the mean absolute error loss function be . The learnable log-variance parameter for the mean squared error loss. Let the mean squared error loss function be . The learnable log-variance parameter of the Huber loss. Here is the Huber loss function.