Message passing graph neural network with vector-scalar message passing and runtime geometric computation

Abstract

A computing system is provided that receives a molecular graph at a messaging graph neural network (MPGNN) and produces scalar embedding representing features of nodes and edges of the graph and vector embedding representing geometric relationships of the graph. The system processes scalar embedding via a vector scalar interactive messaging mechanism of a messaging sub-block of the MPGNN to generate and pass scalar information from the scalar embedding to an embedding space containing the vector embedding. The system updates the vector embedding based on an embedding space containing scalar information and the vector embedding. The system updates scalar embedding based on runtime geometric calculations of geometric relationships encoded in vector embedding. The system calculates an updated molecular map based on the updated scalar embedding and vector embedding, and outputs a target molecular attribute value based on the updated molecular map.

Description

Background Art

[0001] In the field of computational chemistry, computer-based techniques have been developed to predict molecular properties through computer simulations. These molecular properties can have a wide range of effects on the appearance and function of a molecule or material and are therefore of interest in a variety of fields. For example, in the field of drug design, changes in molecular properties can affect the efficacy of a drug. In the field of drug discovery, molecular properties can affect the likelihood that a natural material will be useful for therapeutic purposes. In the field of quantum chemistry, quantum mechanical calculations of the contribution of electrons to the physical and chemical properties of molecules and materials are a fundamental area of ​​inquiry. As discussed below, there is still opportunity to improve computational methods for predicting molecular properties, which will have applications beyond the field of computational chemistry. Summary of the invention

[0002] To solve the problems discussed herein, a computerized system and method are provided. In one aspect, a computerized system includes a processor that receives a molecular graph at a message passing graph neural network (MPGNN) and generates scalar embeddings representing features of nodes and edges of the graph and vector embeddings representing geometric relationships of the graph. The system processes the scalar embeddings via a vector-scalar interactive message passing mechanism of a message passing subblock of the MPGNN to generate scalar information and pass it from the scalar embeddings to an embedding space containing the vector embeddings. The system updates the vector embeddings based on the embedding space containing the scalar information and the vector embeddings. The system updates the scalar embeddings based on runtime geometric calculations of geometric relationships encoded in the vector embeddings. The system calculates an updated molecular graph based on the updated scalar embeddings and vector embeddings, and outputs target molecular property values ​​based on the updated molecular graph.

[0003] This summary is provided to introduce a selection of concepts further described below in the detailed description in a simplified form. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that address any or all of the disadvantages mentioned in any part of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 A schematic diagram of a computing system including a message passing graph neural network (MPGNN) is shown.

[0005] Figure 2 Shows Figure 1 Detailed view of the MPGNN of the computational system.

[0006] Figure 3 Shown by Figure 1 Schematic diagram of the runtime angle calculation of the directional unit vector and the relational position bond angle vector as well as the runtime dihedral angle calculation performed by MPGNN of the computing system.

[0007] Figure 4 Shown by Figure 1 The interatomic distance distribution and density functional theory (DFT) of the system driven MD simulations.

[0008] Figure 5 The energy landscape of Chignolin is shown and Figure 1 Evaluation of energy predictions made by the system, the equivariant transformation (ET) model and the molecular mechanics (MM) model.

[0009] Figure 6 Shows Figure 1 Visualization and model interpretability of the system.

[0010] Figure 7 It is shown Figure 1 Table of the performance of the system showing the mean absolute error (MAE) in energy (kcal / mol) and force (kcal / mol / °A) for 7 small organic molecules in MD17.

[0011] Figure 8 It is shown Figure 1 Table of the performance of the system showing the MAE in rMD17 for energy (kcal / mol) and force (kcal / mol / °A) for 10 small organic molecules.

[0012] Fig.9A Shows Figure 1 A general schematic diagram of the workflow of scalar and vector embedding of the Vector-Scalar Interactive Message Passing (ViS-MP) mechanism of the system.

[0013] Fig. 9B Shows Fig.9A Detailed schematic diagram of the edge-fusion graph attention module with a vector-scalar interactive message passing mechanism.

[0014] Fig.10 Shown by Figure 1 Schematic representation of the potential energy surface (PES) explored by system-driven molecular dynamics simulations and DFT for seven molecules in MD17.

[0015] Fig.11 Shown by Figure 1 System and a plot of the time consumption of a DFT-driven molecular dynamics simulation.

[0016] Fig.12 It is shown Figure 1 Table of the performance of the system showing the MAE of 12 molecular properties on QM9.

[0017] Fig.13Shows Figure 1 Systematic ablation study of aspirin in the MD17 dataset.

[0018] Figures 14 to 16 A flow chart of a computerized method according to one example implementation of the present disclosure is shown.

[0019] Fig.17 An example computing environment is shown according to which embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION

[0020] Computer-based techniques have been developed to predict molecular properties by computer simulation. For example, density functional theory (DFT) is a powerful and widely used quantum physics computing technique that can accurately predict various molecular properties, such as the energy and force of molecules, the shape of molecules, etc. in many cases. However, DFT is time-consuming and computationally intensive, and even for a single model of a simple molecule on a conventional processor, it will usually take several hours. For many complex systems, it is impractical to calculate an accurate DFT solution on current hardware. This currently poses an obstacle to predicting molecular properties.

[0021] Recently, neural network models have been developed for the field of molecular dynamics simulations. Although the accuracy of these models has improved, as described below, these models generally have the disadvantage of high computational cost. Therefore, the widespread application of such neural network models in molecular dynamics simulations faces challenges.

[0022] As atoms change physical positions in the simulation time period, molecular dynamics (MD) model calculates the potential energy and synthetic atomic forces at each atom of the molecular system, thereby describing the dynamics and thermodynamic properties of the molecular system. MD is widely used in physics, chemistry, biology and pharmaceutical fields. Although there is a high computational cost, ab initio MD simulations (such as simulations driven by DFT) can accurately calculate energy and force, thereby limiting the application of such techniques to macromolecular systems and long simulations as described above. In contrast, the classical MD simulations using empirical force fields can achieve fast simulation results for large systems, but suffer from the disadvantages of being unable to capture the quantum effects caused by electronic movement, and the parameters of the force fields calculated in such simulations are usually non-transferable.

[0023] In recent years, deep learning (DL) has demonstrated its powerful ability to learn from raw data without any hand-crafted features in many fields, and DL models for calculating the potential energy of molecular systems have attracted increasing attention. However, an inherent disadvantage of deep learning is that it requires a large amount of data, and this has become an obstacle to its wider application in more scenarios. In order to alleviate the dependence of DL models for calculating potential energy on data, the inductive bias of symmetry can be incorporated into the design of neural networks in a subfield called geometric deep learning (GDL). Here, symmetry describes the conservation of physical laws, that is, the physical properties remain unchanged despite transformations (such as translations or rotations) performed on the underlying data. Due to these limitations, GDL can be extended to only limited data scenarios without the need for data enhancement.

[0024] In GDL, equivariant graph neural networks (EGNNs) have been proposed to model molecular geometry. One type of EGNN for calculating energy potentials uses high-order geometric tensors to achieve equivariance based on group representation theory. Although this method utilizes geometric information, certain operations such as Clebsch-Gordan products (CG products) usually result in extremely large computational overheads at intolerable computational scales, which seriously inhibits the application of this method to macromolecules in practice. In order to alleviate this and improve the modeling of directional information, another method has been to utilize vector embedding and scaling angle representation via the inner product of the vector embedding itself, thereby capturing equivariance in the model. Models such as these models explicitly have the encoded angle and two-face information of the molecular system, which slightly reduces the computational cost by avoiding the CG product operation. However, even these models are subject to relatively high computational costs and leave room for improvement in terms of accuracy. In addition, the disadvantage of these models is that they are not robust to various conformations of the molecular system during molecular dynamics simulations. These models seem unable to effectively utilize such geometric information in message passing, and this may limit their performance.

[0025] To address these issues, a computing system configured to execute a message passing graph neural network MPGNN is provided, which utilizes directional information to achieve high accuracy at low computational cost. As described below, embodiments of the computing system with MPGNN described herein have outperformed prior art methods for molecules in the Molecular Dynamics 17 (MD17) dataset and the revised rMD17 dataset, and have achieved excellent prediction scores for 11 of the 12 quantum properties on the Quantum Machine 9 (QM9) dataset. In addition, simulation results show that the embodiments described herein have demonstrated potential technical benefits that can be extended to protein molecules containing hundreds of atoms, while achieving ab initio accuracy (i.e., accuracy levels close to the true data on which the model was trained) without molecular segmentation. The evaluations and case studies discussed below demonstrate that these embodiments can potentially and effectively explore the conformational space of small and large molecules while providing reasonable interpretability to map geometric representations to molecular structures.

[0026] At a high level, within MPGNN, a runtime geometry computation (RGC) function is executed to extract and encode angle and dihedral information with linear computational complexity, significantly accelerating model training and reasoning and reducing memory consumption. In addition, a vector-scalar interactive message passing mechanism is adopted to effectively utilize geometric information by combining vector hidden representation with scalar hidden representation in an equivariant manner. By incorporating these two modules, MPGNN can achieve high computational efficiency while making full use of geometric information. When fully evaluated on the benchmark, the embodiment of the MPGNN described in this article outperforms the prior art algorithms on all molecules in the MD17 and revised MD17 datasets, and shows excellent performance on the QM9 dataset, indicating the powerful ability of molecular geometry representation. Next, the ab initio molecular dynamics simulation (AIMD) of each molecule on MD17 driven by the MPGNN trained with only 0.7% of the data is discussed. The highly consistent interatomic distance distribution and the explored potential energy surface between AIMD and quantum simulations illustrate that the MPGNN disclosed in the present subject matter is data efficient and can perform simulations with high fidelity. To further explore the scalability of MPGNN for macromolecules, an all-atom MD dataset for the simplest protein Chignolin at the DFT level was constructed, which consists of 9543 different conformations of 166-atom proteins derived from replica exchange molecular dynamics and calculated by DFT. This is considered to be the first MD dataset for real-world all-atom proteins at the DFT level. When evaluated on this dataset, the MPGNN of the present disclosure also achieved excellent performance compared to other deep learning models for predicting potential energy and empirical force fields. In addition, MPGNN is shown to exhibit reasonable interpretability of mapping geometric representations to molecular structures.

[0027] Figure 1 A computing system 10 is shown that includes one or more processors 12 operatively coupled to associated memory 16, data storage device 18, display 20, and communication interface 22 via a data bus 14. The one or more processors 12 are configured to execute a message passing graph neural network (MPGNN) 24 during a training phase to train the MPGNN 24 on a training data set 26, and to execute the trained MPGNN 24 during an inference phase to produce inference results 28. Both the training data set and the inference results may be stored in the data storage device 16.

[0028] During the training phase, i.e., before the inference phase, one or more processors are configured to train on a training data set 26 comprising a plurality of molecular graphs 30 for different conformational geometries of a molecular system 29, and corresponding ground truth values ​​of target molecular properties 38 for each molecular graph 30. Fig.12 As discussed, various target molecular properties 38 may be used. For example, the target molecular properties 38 may be energy parameters, force parameters, or dipole moments. In one example, the target molecular properties 38 may be simultaneously predicted by the MPGNN 24. The ground truth is typically calculated via density functional theory (DFT), but other types of numerical solvers may be used when desired.

[0029] like Figure 1 As shown, the MPGNN 24 includes an embedding block 32, one or more MPGNN processing blocks 34, and an output block 36. Figure 2 A detailed view of the internal structure of each MPGNN processing block 34 is shown in FIG. Figure 2 , each MPGNN processing block 34 includes a corresponding message passing sub-block 40 and a corresponding update sub-block 42. The message passing sub-block 40 is configured with a vector-scalar interactive message passing mechanism 44.

[0030] return Figure 1 During the inference phase, one or more processors 12 are configured to execute the trained MPGNN 24 and feed the inference time input to the MPGNN. To this end, the one or more processors are also configured to receive a molecular graph 30 of a molecular system 29 as an input to the MPGNN at inference time. The molecular graph 30 input at inference time is typically the same type of molecule on which the MPGNN 24 was trained during the training phase, but has a conformational geometry different from the training examples included in the training data set 26. The molecular graph 30 typically includes nodes N connected by edges E. The nodes N represent atoms and the edges E represent interatomic bonds in the molecular system 29.

[0031] The one or more processors 12 are also configured to process the molecular graph 30 using the embedding block 32 to produce a scalar embedding 46 and a vector embedding 48, wherein the scalar embedding 46 encodes scalar information describing the characteristics of the nodes N and the edges E, and the vector embedding 48 represents the geometric relationships between the nodes and edges of the molecular graph, as described in more detail below.

[0032] Brief reference Figure 2 , scalar embeddings 46 include scalar node embeddings 46B and 46C and scalar edge embeddings 46A. Scalar node embeddings 46B, 46C encode the type of atom represented by each node in the molecular graph 30. Scalar target node embedding 46C (h i ) encodes this information for the target node i, and the scalar source node embedding 46B (h j ) encodes this information for each source node j that has an edge ij (i.e., a molecular bond) connected to a target node i. The type of atom may be indicated as a chemical symbol, or another encoding scheme may be used, for example. Scalar edge embedding 46A (f ij ) encodes the interatomic distances represented by each edge E in the molecular graph 30. If desired, other scalar information may also be included, such as the type of bond, functional group labels, etc. These are exemplary types of scalar information included in these embeddings, and it should be understood that other scalar information may also be encoded in the scalar embedding 46. The vector embedding 48 encodes geometric information, which includes a directional unit vector 48A for each node N and a relative position bond angle vector 48 for each of a plurality of node pairs connected by a bond in the molecular graph (i.e., an edge in the molecular graph 30). Other geometric information may also be included in the vector embedding 48. It should be understood that the geometric information describes the angles of the molecular bonds between atoms, while the scalar information describes the distances between atoms and the types of atoms included in the molecular graph 30.

[0033] During inference, initial values ​​for the molecular graph are provided as input, and then the MPGNN processing blocks 34 each update the prediction of the change in the geometry of the molecular graph over discrete time periods. It should therefore be understood that processing the molecular graph using the embedding block includes encoding initial values ​​for scalar node embeddings, scalar edge embeddings, and vector embeddings of the molecular graph via the embedding block. These initial values ​​are then acted upon and updated by the MPGNN processing blocks 34 arranged in successive layers during inference.

[0034] For this purpose, refer to Figure 2, the one or more processors 12 are further configured to process the scalar embedding 46 via the vector-scalar interactive message passing mechanism 44 of the message passing sub-block 40 of the MPGNN 24, thereby generating and passing scalar information from the scalar embedding 46 to an embedding space containing the vector embedding 48. The vector-scalar interactive message passing mechanism 44 includes a scalar message function 44A and a vector message function 44B, and the embedding space is a latent space in a memory accessible by both of these functions 44A, 44B.

[0035] refer to Figure 2 The one or more processors 12 are further configured to update the vector embedding 48 based on an embedding space containing the vector embedding 48 and the scalar information from the scalar embedding 46 via the vector embedding update function 52 of the update sub-block 42, thereby generating an updated vector embedding 48A.

[0036] The one or more processors 12 are also configured to update the scalar embeddings 46 (including the scalar target node embeddings 46C and the scalar edge embeddings 46A) based on runtime geometric calculations of the geometric relationships encoded in the vector embeddings 48 (and more specifically based on their directional unit vectors 48A) via the scalar edge embedding update function 54 and the scalar node embedding update function 56 of the update sub-block 42, thereby producing updated scalar embeddings for the target node (including updated scalar edge embeddings 20A' and updated scalar target node embeddings 20C'), wherein when the source node is the target node, the scalar source node embedding 46B is updated on a separate pass through the update sub-block. The runtime geometric calculations are performed by the runtime geometric calculation function 50 of the update sub-block 42, further details of which are given below. The runtime geometric calculation function 50 includes angle calculation logic 50A and dihedral angle calculation logic 50B. As shown in FIG. Figure 3 As shown, the angle calculation logic 50A is configured to calculate a directional unit vector 48A for the target node and a bond angle vector 48B for each relationship position connecting the target node to the edge of the source node. It should be understood that the directional unit vector 48A is an equivariant vector representation of the change in position and therefore the change in energy of the target node i. Typically, these are only calculated for source nodes within a threshold distance of the target node (i.e., within the neighborhood of the target node). Further as Figure 3 As shown, the dihedral angle calculation logic 50B is configured to calculate the dihedral angle θ between two edges based on the projection of the directional unit vector 48A for each node i, j along the corresponding edge to the nodes m and n.

[0037] return Figure 1The one or more processors 12 are further configured to calculate an updated molecular graph 30A based on the updated scalar embeddings 46A', 46C' and the updated vector embeddings 48' for each node in the molecular graph 30A via the update sub-block 42 by independently treating each node N as a target node.

[0038] return Figure 1 The one or more processors 12 are further configured to output, via an output block 36 , a value of a target molecular property 38 of the molecular system 29 determined based on the updated molecular graph 30A.

[0039] As shown, the value for the target molecular property 38 can be output to a downstream program 39 along with the updated molecular graph 30A for further processing, and to the data storage device 16 as an inference result 28. As an example, the downstream program 39 can be a molecular dynamics simulation program for use in a molecular dynamics simulation 39A over multiple time steps. For example, the potential energy of each node in the updated molecular graph 30A can be calculated, and the potential energies of all nodes can be summed to calculate the total potential energy as the target molecular property 38. Similarly, interatomic forces can be calculated along each edge based on the potential energy at each node, as described below.

[0040] One or more processors 12 are configured to process the scalar embedding 46 via the vector-scalar interactive messaging mechanism 44 of the messaging sub-block 40 at least in part by the following operations: for each MPGNN processing block 34, for each target node among the multiple target nodes in the molecular graph 30, in the MPGNN processing block loop: via the scalar message function 44A of the messaging sub-block, generate an edge-side scalar message 58 for each edge connected to the target node, pass the edge-side scalar message 58 to the vector message function 44B of the messaging sub-block 40, and based on the vector embedding 48 and the edge-side scalar message 58 from each edge connected to the target node, generate an aggregated vector message 60 via the vector message function 44B, wherein the vector embedding 48 and the edge-side scalar message 58 are aggregated together for the target node via the vector scalar aggregation function 62. Thus, the one or more processors 12 may be configured to aggregate the vector embedding and scalar embedding for each target node, respectively, across the source nodes connected to the target node, to generate a corresponding aggregated scalar message and aggregated vector message for the target node in a MPGNN processing block loop prior to updating the vector embedding and scalar embedding. The scalar message encoding information includes or is based on one or more (and typically all) of the following: scalar node embeddings for the target node and adjacent (i.e., connected and within a threshold distance) source nodes, a scalar edge embedding for the target node, and a calculated attention score for one or more scalar node embeddings and scalar edge embeddings for the target node from a trained graph attention network.

[0041] The one or more processors are configured to generate a scalar message at least in part by fusing a scalar node embedding and a scalar edge embedding to generate a fused scalar embedding. Fusion can be achieved by concatenation, Hadamard product, or adding a learnable bias term or other suitable technique. In addition, calculating the attention score can be calculated based on the fused scalar embedding via a nonlinear activation function or other suitable activation function.

[0042] In the MPGNN processing block loop, one or more processors may be configured to update the vector embedding via an update sub-block at least in part by the following operations: updating the vector embedding for the target node based on the aggregated scalar message and the aggregated vector message for the target node via the update sub-block.

[0043] In the MPGNN processing block loop, one or more processors are configured to update the scalar embedding via an update sub-block at least in part by the following operations: via the update sub-block, perform runtime geometry calculations to calculate runtime values ​​of relative position bond angle vectors, direction unit vectors, and dihedral angles for each target node; via the update sub-block, calculate an updated scalar node embedding for the target node based on the calculated relative position bond angle vectors, aggregated scalar messages for the target node, and scalar node embeddings for the target node; and via the update sub-block, calculate an updated scalar edge embedding based on the calculated dihedral angles and scalar edge embeddings. One or more processors may be configured to calculate an updated molecular graph based on the updated scalar edge embeddings, the updated scalar node embeddings, and the updated vector embeddings.

[0044] The following paragraphs provide additional description of implementation details of a particular embodiment of the MPGNN 24, referred to as a Vector Scalar Interactive Graph Neural Network (ViSNet), which may be implemented by one or more processors 12 of the computing system 10 described above. In the following paragraphs, reference is generally made to Figure 1 to Figure 2 , where reference notation is used sparingly to avoid confusion with the mathematical notation for embeddings, vectors, functions, and their output parameters. References to ViSNet hereinafter should be understood to refer to a specific implementation of MPGNN 24, and similar terminology is used to describe similar components as described above. At a high level, ViSNet embeds the 3D structure of a molecule, and geometric information is extracted through a series of ViSNet blocks, and energies and forces are output through output blocks. The energies and forces predicted by ViSNet can be used to drive molecular dynamics simulations. References Figure 2As briefly discussed above, the ViSNet block consists of two sub-blocks: a message sub-block and an update sub-block. These two sub-blocks formulate a vector-scalar interactive message passing mechanism, one implementation of which is referred to herein as ViS-MP, which implements Equations 5 to 9. Specifically, in the message sub-block, first, the node embedding h i The message function φ m , for each edge connecting target node i to source node j, the edge embedding f ij , relative position vector (also called the relative position bond angle vector, which is the vector between the target node i and each source node j) and the direction unit vector To get the scalar message m ij and vector message Then, the scalar message m ij and vector message is aggregated to the target node i. In the update sub-block, the aggregate scalar message m i The output of the angle calculation logic of the runtime geometry calculation function is updated by the function from Update i Then, the output of the dihedral angle calculation logic of the runtime geometry calculation function is updated by the update function from and Update f ij . Figure 3 Shows the angles calculated by the runtime geometry calculation functions. Figure 2 ,Finally, the vector embedding Both scalar and vector messages are passed through the update function Update. The overall design of ViS-MP aims to improve the interaction between scalar embedding and vector embedding. Figure 3 A diagram of directional geometric information extracted by an RGC function with linear complexity is shown. The RGC function includes two sets of programming logic - angle calculation logic and dihedral angle calculation logic. In the RGC function, an equivariant vector representation for each node (referred to herein as a "directional unit vector") is designed to preserve the geometric information for the node. Figure 3 By performing the rejection and inner product between the two directional unit vectors of nodes i and j in (see Equations 1 to 4), the angle and dihedral information can be directly obtained, where the computational complexity O(N) for the angle and dihedral calculation is low.

[0045] ViSNet is a general GDL model for predicting energy potential, which can predict potential energy, atomic forces, and various quantum chemical properties by taking atomic coordinates and atomic numbers as input. Figure 1As shown, the ViSNet model consists of an embedding block and multiple stacked ViSNet blocks, followed by an output block. The atomic number and coordinates are fed into the embedding block, followed by a ViSNet block to extract and encode the geometric representation. The geometric representation is then used to predict molecular properties through the output block. It is worth noting that ViSNet is an energy-efficient potential model, i.e., the predicted atomic forces are derived from the negative gradient of the potential energy with respect to the atomic coordinates.

[0046] like Figure 2 As shown and briefly discussed above, each ViSNet block consists of a message sub-block and an update sub-block. These blocks work together as part of a vector-scalar interactive message passing mechanism (called ViS-MP). The rich geometric information embedded in the messages passed via ViS-MP is extracted by the RGC function with linear complexity. The RGC function and ViS-MP will be explained in detail in the following paragraphs.

[0047] Turning now to the RGC function, the success of classical force fields suggests that geometric features such as interatomic distances, angles, and dihedral angles can be used to determine the total potential energy of a molecule. The explicit extraction of invariant geometric representations in existing methods often suffers from the drawback of consuming a lot of time or memory during model training and inference. Given an atom, the computation of angle information scales with the number of neighboring atoms in the square of N (N 2 ), while the computational complexity for dihedral angles is N cubed O(N 3 ). To alleviate this problem, the RGC function is proposed, which uses an isovariant vector representation called a directional unit vector for each node to preserve its geometric information. RGC calculates the geometric information directly from the directional unit vector, which only sums the vectors from the target node i to its adjacent source node j once. Therefore, the computational complexity can be reduced to O(N).

[0048] consider Figure 3 The substructure of the example molecule with four atoms shown in can be obtained from the vector Get the angle information of target node i: in is the vector from node i to its adjacent source node j, yes Here, the direction unit vector of node i is It is proposed as the sum of all unit vectors from node i to all its neighboring nodes j, where node i is the intersection of all unit vectors. As shown in Equation 2, the direction unit of node i is calculated The inner product of , which represents the sum of the inner products of the unit vectors from node i to all its neighboring nodes. Combined with Equation 1, the direction The inner product of finally represents the sum of the cosine values ​​of all angles formed by node i and any two of its neighbors.

[0049] Similar to the runtime angle calculation, the vectors are calculated separately and The direction unit of node i on and the direction unit of node j Vector rejection. in Indicates perpendicular to of The vector component of is called vector rejection. and is defined in Eq. 1. Rejection vector The sum of Rejection vector Then calculate and The inner product between is carried out as follows The dihedral information of the axis: By calculating the vector With vector The inner product of can obtain the cosine value of all dihedral angles and eij as the common rotation axis, such as Figure 3 Note that directional unit vectors are not limited to Cartesian coordinates, but can be extended to higher-order tensors via spherical harmonics.

[0050] Now turning to the details of ViS-MP, in order to effectively utilize geometric information and enhance the interaction between scalars and vectors, a vector-scalar interactive message passing mechanism (ViS-MP) is designed with respect to intersection nodes and edges for angles and dihedrals, respectively. The following operations are performed by ViS-MP: where h i represents the scalar embedding of node i, f ij Represents the edge feature between node i and node j. represents the embedding of the directional unit vector mentioned in the description of the RGC function above. The superscript of the variable indicates the index of the block to which the variable belongs. ViS-MP uses vector-scalar interaction to extend the conventional message passing, aggregation, and update process. Equations 5 and 6 depict the message passing and aggregation process. Specifically, the scalar embedding h is incorporated j 、h i and f ijThe scalar message mij is passed and then passed through the message function is aggregated to node i (Equation 5). A similar operation is applied to the vector message of node i It contains the scalar message m ij , Vector and vector embedding (Equation 6). Equations 7 and 8 show the update process. As shown in the figure, h i Output m by aggregating scalar messages i To update, and at the same time through the update function To update Then, we embed and The inner product of the rejection is updated by the function To update Finally, the vector embedding Both scalar and vector messages are passed through the update function Update. It is worth noting that for the nonlinear function of the vector (i.e., φ v ) are equivariant. Details about the message and update functions can be found below.

[0051] Reference now Fig.9A , a more complete description of the operation of ViS-MP will now be explained. Fig.9A The general workflow of scalar and vector embedding is shown in Figure 2. Specifically, a ViSNet block consists of two modules: i) Scalar2Vec, which appends a scalar embedding to a vector; ii) Vec2Scalar, which repairs the scalar embedding built by the RGC function. The input of Scalar2Vec is the node embedding h i , edge embedding f ij , direction unit vector and relative positions. The edge fusion graph attention module (which is used as ) ij The output of the subsequent dense layer is used as input h i , and outputs a scalar message. Before aggregation, each scalar message passes through a dense layer and is associated with a relative position unit vector and its own direction unit vector Fusion. A dense layer is a layer that is deeply (fully) connected to its previous layer (i.e., the output of the edge fusion graph attention module), and is used to change the dimension of the output of the previous layer by performing matrix-vector multiplication. Then, the vector message is calculated and the calculated vector messages between neighbors are aggregated. Through the gated residual connection, the final residual In the Vec2Scalar module, the final Δh is calculated by taking the Hadamard product of the aggregated scalar message and the output of the angle calculation logic of the RGC function and adding a gated residual connection i Similarly, the projection of the logit is calculated by combining the dihedral angles of the RGC function ij and output to determine the final Δf ij .

[0052] In summary, geometric features are extracted by taking the inner product of the output of the RGC function, and the scalar embedding and vector embedding are cyclically updated with each other in ViS-MP to learn a comprehensive geometric representation from the molecular graph.

[0053] ViSNet can be used to make accurate quantum chemical property predictions. As its evidence, ViSNet has been evaluated on several major benchmark data sets including MD17, revised MD17 (called "rMD17") and QM9 for energy, force and other molecular properties predictions. MD17 consists of MD trajectories of 7 small organic molecules, and the number of conformations in each molecular data set ranges from 133,700 to 993,237. Data set rMD17 is a re-creation version of MD17 with higher accuracy. QM9 consists of 12 kinds of quantum chemical properties of 133,385 small organic molecules with up to 9 heavy atoms. ViSNet is compared with the results of other prior art algorithms for molecular property prediction, including kernel-based algorithms FCHL19 and GAP, algorithms SchNet, ANI, PhysNet, EGNN, ACE, DimeNet / DimeNet++, GemNet, PaiNN and ET based on directional information, and algorithms UNiTE and NequIP based on group representation theory. The training details of ViSNet on each benchmark are described below.

[0054] like Figure 7 Table 1 and Figure 8As shown in Table 2, it is significant that ViSNet is superior to the algorithms compared with it for all molecules with the lowest mean absolute error (MAE) for both predicted energy and force. Table 1 shows the mean absolute error (MAE) for energy (kcal / mol) and force (kcal / mol / °A) of 7 small organic molecules on MD17 compared with the prior art algorithms. The best results in each category are highlighted in bold. Table 2 shows the mean absolute error (MAE) for energy (kcal / mol) and force (kcal / mol / °A) of 10 small organic molecules on rMD17 compared with the prior art algorithms. The best results in each category are highlighted in bold. Although only 950 samples of each molecule are used to train ViSNet, and another 50 samples are used in the validation set, ViSNet is still far superior to kernel-based algorithms, which indicates that the equivariant model design in ViSNet efficiently captures geometric information, and therefore significantly alleviates the requirements of a large number of training samples. On the one hand, ViSNet incorporates more directional geometric information through the strategy of RGC function compared with PaiNN and ET, which seems to contribute to the performance gain. On the other hand, considering the adoption of angle and dihedral information in Gem-Net, the superior performance of ViSNet over Gem-Net indicates that ViS-MP can better utilize geometric information during message passing to achieve improved results.

[0055] In addition, ViSNet also achieved excellent performance in predicting the quantum chemical properties of QM9. Fig.12 The Extended Data Table 1 shows the mean absolute error (MAE) of 12 molecular properties on QM9 compared to state-of-the-art algorithms. The best results in each category are highlighted in bold. As shown, ViSNet outperforms the compared algorithms for 11 of the 12 chemical properties and achieves comparable results for the remaining properties.

[0056] Molecular dynamics simulations are a useful application of the predicted potential energies and atomic forces from ViSNet. To evaluate ViSNet as a potential energy prediction model for ab initio molecular dynamics simulations, an instance of ViSNet was created that was trained using only 0.7% of the samples available on MD17 in the ASE simulation framework (i.e., 950 samples for model training) to perform ab initio MD simulations for all seven organic molecules. In this analysis, all simulations were run at a Berendsen thermostat with a time step of τ = 0.5 fs, and the other settings were the same as those for the MD17 dataset.

[0057] Figure 4Figure 2 shows the interatomic distance distributions of molecular dynamics simulations driven by ViSNet and DFT. At (a), a plot of the atomic density at a radius r centered on a random atom is shown. The interatomic distance distribution h(r) is defined as the ensemble average of the atomic density at radius r. Figure 2 At (bh) it is shown that the distribution derived from ViSNet is very close to the distribution generated by DFT. The interatomic distance distribution is defined as the overall average of the atomic density. At (b) to (h), a comparison of the interatomic distance distribution between ViSNet and DFT simulations is shown for all seven organic molecules in MD17. For all 7 molecules, the interatomic distance distributions shown were derived from AIMD simulations with ViSNet as the potential energy model and from ab initio molecular dynamics simulations at the DFT level. The results for ViSNet are shown using solid lines, while dashed lines are used for the DFT results. It can be seen that the curves generated by the DFT and ViSNet results are almost indistinguishable. The structures of the corresponding molecules are in Figure 4 are shown in the upper right corner of each panel (b)-(h).

[0058] exist Fig.10 The potential energy surfaces sampled by ViSNet and DFT for these molecules are compared in Fig. Fig.10 The potential energy surface (PES) probed by molecular dynamics simulations driven by ViSNet and DFT for 7 molecules in MD17 is shown. The potential energy surface is drawn to allow the conformational population to be examined from simulations driven by DFT and ViSNet. The snapshots in each simulation trajectory are aligned with the initial structure of the corresponding molecule in MD17. Then, principal component analysis (PCA) is applied to the coordinates of each conformation. PC1 and PC2 have been set as two axes. The potential energy value is calculated as ΔG(x, y)=k B Tlng(x, y), where k B is the Boltzmann constant, T is the temperature of the system, and g(x, y) represents the normalized joint probability distribution. The minimum energy value is set to zero. 100 bins are applied to generate the landscapes on the x and y axes. In each panel, the left and right sub-figures show the PES of the same molecule obtained by DFT and ViSNet, respectively.

[0059] Fig.10 The consistent potential energy surfaces shown indicate that ViSNet can recover dynamical properties and conformational space well from simulation trajectories, which demonstrates the usefulness of ViSNet for realistic molecular dynamics simulations. In addition, compared with the excessive computational cost of DFT, ViSNet significantly saves 2-3 orders of magnitude of computational time, as shown in Figure 2. Fig.11 and Fig.13The expanded version of is shown in Table 2. These results demonstrate that, with only a few training samples, ViSNet can act as a potential energy prediction model to perform high-fidelity molecular dynamics simulations with much less computational cost.

[0060] Fig.11 The time spent performing molecular dynamics simulations driven by ViSNet and DFT is shown. For 7 molecules in MD17, the simulation was run using the ASE framework. The average time spent for each simulation step is recorded in seconds. The y-axis is a logarithmic axis with a base of 10. As shown in the figure, ViSNet calculations have significantly lower average time per step than DFT calculations.

[0061] Figure 5 The energy landscape of Chignolin and the visualization of the evaluation of the energy prediction performed by ViSNet, equivariant transformation (ET) and molecular mechanics (MM) methods are shown. At (a), the energy landscape of Chignolin sampled by REMD is shown. The x-axis of the landscape is the distance between the main chain O on Y2 and the main chain N on G6, and the y-axis is the distance between the main chain O on E4 and the main chain N on T7. Six representative structures are then selected for visualization. Each structure is shown as a cartoon, and residue is depicted with a short straight line. The histogram shows the mean absolute error (MAE) between the energy difference predicted / calculated by ViSNet, ET, MM and the ground truth calculated by DFT for the corresponding structure. At (b) to (d), the energy correlation on the test data set between the ground truth calculated by DFT and the prediction performed by ViSNet, ET and molecular mechanics is shown. The corresponding distribution and the ground truth of energy prediction or calculation are shown in each panel.

[0062] ViSNet can be applied to real-world proteins to explore its scalability from small organic molecules to large biomolecules. Considering that the time complexity of DFT is roughly N cubic with the number of atoms O(N 3), so the simplest protein Chignolin with 166 atoms is used to construct a DFT-level MD data set for model training and evaluation. For data generation, 80ns replica exchange molecular dynamics (REMD) simulations are run to sample various folded and unfolded states of Chignolin. As a result, 9,543 representative conformations are collected, and the energy and force on the nucleus are calculated by the Gaussian 16 software package. It is believed that this is the first MD data set of real-world all-atom proteins at the DFT level. The data generation process is described in detail below. The Chigolin data set is split into a training set, a validation set, and a test set at a ratio of 8:1:1. ViSNet and the models compared therewith are trained to have the best performance in the evaluation detailed below, including ET, NequIP and GemNet on the Chignolin data set and its default settings on the Tesla V100 GPU. During model training, even if the batch size is set to 1, GemNet fails due to GPU memory exhaustion, and NequIP suffers from underfitting using its default hyperparameters on the Chignolin data set. ViSNet and ET can be successfully trained and compared with molecular mechanics (MM). DFT results are used as the ground truth. Figure 5 At (a) is shown the REMD sampling and d Y2-G6 (the distance between the main chain O on Y2 and the main chain N on G6) and d E4-T7 The free energy landscape of Chignolin described by (the distance between the main chain O on E4 and the main chain N on T7). The concentrated energy basin on the left shows the folded state, and the dispersed energy basin on the right shows the unfolded state. Six representative structures are selected in the low potential energy region with folded and unfolded states, and some intermediate states with high potential energy are selected. The energy prediction for the six representative structures is visualized, and ViSNet produces a significantly better potential energy estimate than any one of the ET trained on the same data set and the MM with empirical force field. Figure 5 The correlation between the energies predicted by ViSNet, ET, and MM and the ground truth given by DFT for the conformations in the test set is shown at (b) to (d). ViSNet achieves the lowest MAE and the highest R2 score. These results show that ViSNet has the ability to scale to real-world proteins with a small training set and achieves excellent accuracy and efficiency.

[0063] To further explore where the performance gain of ViSNet comes from, a comprehensive ablation study is performed. Specifically, the runtime angle calculation logic (w / o A), the runtime dihedral angle calculation logic (w / o D), and both (w / o A&D) are excluded in the ablation study of ViSNet performance in order to evaluate the usefulness of each part. In addition, some model variants are designed with different message passing mechanisms for scalar and vector interactions based on VIS-MP. For example, ViSNet-N is designed to aggregate dihedral information directly to intersection nodes, and ViSNet-T is designed to utilize another form of dihedral angle calculation. The results of the ablation study are shown in Fig.13 In the extended table 2 of . Specifically, extended table 2 shows the results of the ViSNet ablation study of aspirin in the MD17 dataset. Extended Table 2 lists the results of ViSNet and its variants without runtime angle calculation (w / o A), without runtime dihedral angle calculation (w / o D) and without any of them (w / o A&D), and compared with the message passing variants ViSNet-N and ViSNet-T. The best results are shown in bold. Based on the results, it can be concluded that both types of directional geometric information are useful, and the dihedral angle information contributes to the final performance. In addition, the significant performance degradation from ViSNet-N and ViSNet-T further verifies the effectiveness of the ViS-MP mechanism.

[0064] We will now discuss the visualization and interpretability of molecular structures with ViSNet. To explain how incorporating geometric features such as angles and dihedrals improves the expressiveness of ViSNet, we illustrate the model for the interpretability of ViSNet by mapping the angle and dihedral representations derived from the inner product of the directional unit vectors in the model to the atoms and bonds of the molecular structure. This can bridge the gap between the geometric representation in ViSNet and the molecular structure. and The embeddings are visualized after the inner product of . The high-dimensional embeddings are reduced to a two-dimensional space using T-SNE and then clustered using DBSCAN without a priori on the number of clusters.

[0065] Figure 6 The clustering results of the embeddings after taking the inner product of the directional unit vectors are shown at (a) and (b). Figure 6 In (d) and The clustering result is mapped back to the chemical structure of aspirin, and Figure 6 Aspirin atoms are shown at (c) with indices marked. Interestingly, the embeddings for both the intersection nodes and edges can be clearly clustered into several clusters shown in different colors. For example, although the carbon atom C 11 and carbon atom C12 have different positions and are connected to different atoms, but their inner product are clustered into the same class to preserve similar substructures ({C –1 -O2O3C6} and {C –2 -O1O4C 13}).

[0066] Figure 6 It is a visualization to evaluate the model interpretability of ViSNet. Figure 6 At (a) it is shown Clustering of the embeddings afterwards. It represents the angle with the intersection node i as the vertex. Figure 6 At (b) it is shown Clustering of the embeddings afterwards. Represents the intersection edge e ij is the dihedral angle representation of the common rotation axis. Figure 6 The chemical structure of aspirin is shown at (c). The carbon and oxygen atoms are shown as darker shades. Figure 6 The chemical structure of aspirin is shown at (d) with atoms shaded according to their clustering in panel (a). Figure 6 The chemical structure of aspirin is shown at (e), where the bonds are shaded according to the clustering in panel (b). For simplicity, hydrogen atoms in the chemical structure of aspirin in panels (c) to (e) are omitted.

[0067] In order to Mapping to the edges in the aspirin chemical structure, more clusters are detected because the distance between two atoms within the threshold is taken into account. Figure 6 The panel shown in (b) shows only the two largest clusters, corresponding to Figure 4 (e) Aromatic bonds and CO bonds. Intuitively, {C8, C 10}、{C 10 , C9} and {C9, C7} have similar structures ({C6-(C8-C 10 )-C9}, {C8-(C 10 -C9)-C7} and {C 10 -(C9-C7)-C5}). In addition, even if {O2, C 11} and {O3, C 11} belong to different types of chemical bonds, the intersecting edges are still classified into the same category because they have similar structures ({(O2-C 11 )-C6O3} and {(O3-C 11)-C6O2}). Hence, from this visualization, we can conclude that ViSNet can distinguish different molecular substructures in the embedding space.

[0068] Now turn to the detailed operation of ViSNet. ViSNet starts from the current state of atoms (including atomic positions and atomic number ) predict molecular properties (e.g., energy force The proposed ViSNet architecture is Figures 1 to 3 The overall design of ViSNet follows the vector-scalar interactive message passing shown in Equations 5 to 8. First, the embedding block encodes the atomic number and edge distance into the embedding space. Then, a series of ViSNet blocks update the node-by-node scalar and vector representations based on their interactions. A residual connection is placed between two ViSNet blocks. Finally, the stacked corresponding gated equivariant blocks are attached to the output block for specific target molecular property prediction.

[0069] In the embedding block, ViSNet extends direct node and edge embeddings with their neighbors. First, the atomic chemical symbol z is embedded i , and compute an edge representation with a cutoff distance through the radial basis function (RBF). It will be appreciated that for computational efficiency purposes, the RBF cuts off edges beyond a threshold distance (usually expressed in angstroms). Then, atom i, its 1-hop neighbor j, and the directly connected edge e within the cutoff ij The initial embeddings of are fused together as the initial node embedding and edge embedding In summary, the embedding block is given by:

[0070] N(i) represents the set of 1-hop neighboring nodes of node i, and j is one of its neighbors. Initial vector embedding is set to Vector Embedding is projected into the embedding space; and F is the size of the hidden dimension. The advantage of this projection is to assign a unique high-dimensional representation to each embedding to distinguish one from another. Further discussion on its effectiveness and interpretability is given in the Results section.

[0071] refer to Fig.9A , in the Scalar2Vec module, vector embedding It is updated by both scalar messages derived from node and edge scalar embeddings (Eq. 5) and vector messages with inherent geometric information (Eq. 6). The message of each atom is calculated by an edge fusion graph attention module, which fuses node embeddings and edge embeddings and calculates the attention score. The fusion of node embeddings and edge embeddings can be a cascade operation, Hadamard product, or adding a learnable bias. Graph attention networks with multi-head attention mechanisms can be used to implement edge fusion using Hadamard products.

[0072] Briefly go to Fig. 9B , showing Fig.9A Detailed view of the edge fusion graph attention module of . The edge fusion graph attention module implements a graph attention network that combines features of a graph neural network and one or more attention layers. As shown, the edge fusion graph attention module includes a multi-headed attention mechanism that takes as input a scalar target node embedding 46B (as the Q value of the attention mechanism), a scalar source node embedding 46C (as both the K value and the V value of the attention mechanism), and a scalar edge embedding 46A (as both the K value and the V value of the attention mechanism), and the output of an RBF, the output of which is a scalar valued cosine cutoff of the magnitude of the radial vector along edge ij. The fusion representation is passed through a non-linear activation function, such as a Sigmoid Linear Unit (SiLU) function as shown in Equation 11. The values ​​(V) in the attention mechanism of the graph attention network implemented by the edge fusion graph attention module are also fused with the edge features before being multiplied by the attention score weighted by the cosine cutoff, as shown in Equation 12, where l∈{0, 1, 2, …, L} is the index of the block, σ represents the activation function (e.g., SiLU), W is the learnable weight matrix, ⊙ represents the Hadamard product, φ(·) represents the cosine cutoff, and Dense(·) refers to a learnable weight matrix with the activation function. For simplicity, the learnable bias is omitted for the linear transformation of the scalar embedding in the equation, and there is no bias for the vector embedding to ensure universal equivariance. Equations 11 and 12 are in Fig. 9B This is shown graphically in , where the output of Equation 12 is passed to the downstream dense layer.

[0073] return Fig.9A , then, the calculated Used to generate geometry messages for vectors And the vector embedding Updated by:

[0074] continue Fig.9A, in the Vec2Scalar module, the node embeds and edge embedding The geometric information extracted by the RGC strategy, i.e., angle (Eq. 7) and dihedral angle (Eq. 8), are updated respectively. Residual Node Embedding It is calculated by the Hadamard product between the runtime angle information and the aggregated scalar message with a gated residual connection: To compute the residual edge embedding Perform the Hadamard product of the runtime dihedral information with the transformed edge embeddings: After computing the residual hidden representation, the erased hidden representation is added to the original input of block l and fed to the next block.

[0075] In the output block, multiple gated equivariant blocks are used to update the scalar and vector embeddings of the nodes: where [·,·] is a tensor concatenation operation. The final scalar embedding and vector embedding Used to predict various molecular properties.

[0076] On QM9, the molecular dipole is calculated as follows: in represents the center of mass. Similarly, for the electronic space range <R 2 >, using the following equation: For the remaining 10 properties y, the final scalar embeddings of the nodes are aggregated as follows: For models trained on molecular dynamics datasets including MD17, revised MD17, and Chignolin, the total potential energy is obtained as the sum of the final scalar embeddings of the nodes. As the energy-saving potential, the forces are then calculated using the negative gradient of the predicted total potential energy with respect to the atomic coordinates:

[0077] The design of the Chignolin data set will now be described. The initial structure for replica exchange molecular dynamics (REMD) simulations is derived from the Protein Data Bank (PDB ID: 5AWL). Water molecules are removed from the crystal structure. Then, the FF19SB force field is applied to describe the atomic interactions for Chignolin in the generalized Born implicit solvent model. The second modification of the Bondi Van der Waals radius set is used in the solvent model. The program CHIR_RST in Aber20 is applied to create a chiral constraint file during the REMD simulation to maintain chiral properties at high temperatures. The system encounters a minimization process of 500 steepest descents and 500 conjugate gradient cycles at the beginning. After energy minimization, a 200ps equilibrium run at 300K, 400K, 500K, 600K, 700K, 800K, 900K, 1000K is applied to the system with a random initial speed. The final structure of the equilibrium is used for REMD simulations at the corresponding temperature. Each single replica in the production runs for 2ps and then exchanges to the adjacent temperature. The exchange occurred 5,000 times in each production run and 8 replica temperatures were obtained, which resulted in a total simulation time of 80 ns. The sampling interval for each simulation trajectory was 0.4 ps, so the trajectory had 200,000 points. 10,000 points were uniformly picked from the REMD trajectory to generate the input file for Gaussian 16. The potential energy and atomic forces for each conformation were calculated using the M06-2X functional and the 6-31G* basis. The integration grid was set to ultrafine accuracy.

[0078] Finally, 9,543 SCF convergent conformations with total potential energy and atomic forces were recruited from the Chignolin dataset. The distribution of total energy ranges from -2,831,076.155 kcal / mol to -2,830,477.983 kcal / mol, and some representative conformations are shown in Supplementary Table 1. Figure 2 Note that the total energy does not show a normal distribution, but has two peaks corresponding to the folded and unfolded states of Chignolin, which increases the difficulty of model training on the dataset.

[0079] Regarding the data splitting scheme, for the QM9 dataset, the dataset was randomly split into 110,000 samples as the training set, 10,000 samples as the validation set, and the rest as the test set by following previous studies. To evaluate the effectiveness of ViSNet on simulated data, ViSNet was trained on MD17 and rMD17 with a limited data setting consisting of only 950 uniformly sampled conformations for model training and 50 conformations for validation of each molecule.

[0080] In addition, the entire Chignolin dataset was randomly split into 80%, 10%, and 10% as training dataset, validation dataset, and test dataset. For illustration, six representative conformations were selected from the test set.

[0081] About experimental setting, for QM9 data set, batch size of 32 and learning rate of 1e-4 were adopted for all properties. Model training was carried out using mean square error (MSE) loss. For molecular dynamic data sets including MD17, rMD17 and Chignolin, combined MSE loss for energy and force prediction was utilized. For MD17 and rMD17, the weight of energy loss was set to 0.05, while for Chignolin, the weight of energy loss was set to 0.2. For MD17 and rMD17, the weight of force loss was set to 0.95, while for Chignolin, the weight of force loss was set to 0.8. For different molecules, batch size was set to 4, and learning rate was selected from 2e-4, 3e-4, 4e-4. For small molecules in QM9, MD17 and rMD17, cutoff was set to 5, while for Chignolin it was changed to 4, so as to reduce the number of edges in the molecular graph. If the validation loss stops decreasing, learning rate decay is used. Patience was set to 15 epochs for QM9 and 30 epochs for MD17, rMD17, and Chignolin. The learning rate decay factor was set to 0.8 for these models. An early stopping strategy was adopted to prevent overfitting. The ViSNet model trained on the Molecular Dynamics dataset has 9 hidden layers and the embedding dimension is set to 256. A larger model was used for the QM9 dataset, i.e., the embedding dimension became 512. The experiments were conducted on a 32G-V100 GPU.

[0082] Figures 14 to 16 A flowchart of a computerized method 100 according to an example implementation of the present disclosure is shown. 100 may be implemented by the hardware and software of the computing system 10 described above, or by other suitable hardware and software.

[0083] At 102, method 100 includes training a message passing graph neural network (MPGNN) on a training data set during a training phase prior to an inference phase, the training data set including a plurality of molecular graphs for different conformational geometries of a molecular system and corresponding ground truth values ​​of a target molecular property for each molecular graph. As shown, the target molecular property may be an energy parameter 102A, a force parameter 102B, or a dipole moment 102C. As described above, other parameters may also be considered.

[0084] At 104, the method 100 includes executing a message passing graph neural network (MPGNN) via one or more processors of a computing device. At 106, the method 100 also includes receiving a molecular graph of a molecular system as an input of the MPGNN. A molecular graph generally includes nodes connected by edges, the nodes representing atoms, and the edges representing bonds between atoms in the molecular system.

[0085] At 108, method 100 includes processing a molecular graph using an MPGNN to generate a scalar embedding and a vector embedding, the scalar embedding encoding scalar information describing features of nodes and edges, the vector embedding representing geometric relationships between nodes and edges of the molecular graph. As shown at 110, the scalar embedding may include a scalar node embedding and a scalar edge embedding. The scalar node embedding may encode the type of atom represented by each node, and the scalar edge embedding may encode the interatomic distance represented by each edge. The vector embedding may encode geometric information including a directional unit vector for each node in the molecular graph and a relative position bond angle vector for each node pair in a plurality of node pairs. At 112, processing the molecular graph at 108 is shown as including encoding initial values ​​for the scalar node embedding, the scalar edge embedding, and the vector embedding for the molecular graph.

[0086] from Fig.14 Continue to step 112 in Fig.15 Step 114 in FIG. 1 shows a MPGNN processing block loop, which starts at step 114 and continues until Fig.16 At step 138 in the process, the method 100 loops back until the last MPGNN processing block is reached. Therefore, the method 100 can also execute steps 114-138 in the MPGNN processing block loop for each target node in the multiple target nodes in the molecular graph for each MPGNN processing block.

[0087] At 114, method 100 includes processing the scalar embedding via a vector-scalar interactive message passing mechanism of the MPGNN, thereby generating scalar information and passing it from the scalar embedding to an embedding space containing the vector embedding. Sub-steps of the processing at 114 are shown at 116-122. At 116, processing the scalar embedding via the vector-scalar interactive message passing mechanism at 114 is shown as being implemented at least in part by generating a scalar message via a scalar message function of the MPGNN, the scalar message encoding information based on one or more of a scalar node embedding for a target node and adjacent source nodes, a scalar edge embedding for the target node, and a calculated attention score for one or more scalar node embeddings and scalar edge embeddings for the target node from a trained graph attention network. At 118, it is shown that generating the scalar message can be implemented at least in part by fusing the scalar node embedding and the scalar edge embedding to thereby generate a fused scalar embedding, the fusion being implemented by concatenation, Hadamard product, or adding a learnable bias term, and calculating the attention score based on the fused scalar embedding via a non-linear activation function. Other techniques for fusing embeddings can also be applied, as well as other activation functions that preserve equivariance.

[0088] Processing the scalar embedding at 114 may include passing the scalar message to a vector message function of the MPGNN at 120 , and generating a vector message via the vector message function based on the vector embedding and the scalar message at 122 .

[0089] At 124, in the MPGNN processing block loop, method 100 further includes, before updating the vector embedding and the scalar embedding, respectively aggregating, for each target node, the vector embedding and the scalar embedding across source nodes connected to the target node to generate a corresponding aggregated scalar message and an aggregated vector message for the target node.

[0090] At 126, method 100 includes updating the vector embedding based on the embedding space containing the vector embedding and the scalar information from the scalar embedding. At 128, updating the vector embedding is accomplished at least in part by updating the vector embedding for the target node based on the aggregated scalar message and the aggregated vector message for the target node.

[0091] At 134, method 100 includes updating the scalar embedding based on runtime geometry computation of the geometric relationship encoded in the vector embedding. At 132, updating the scalar embedding is performed at least in part by performing runtime geometry computation to compute runtime values ​​of a relative position bond angle vector, a direction unit vector, and a dihedral angle for the target node; computing an updated scalar node embedding for the target node based on the computed relative position bond angle vector, an aggregated scalar message for the target node, and a scalar node embedding for the target node; and computing an updated scalar edge embedding based on the computed dihedral angle and the scalar edge embedding.

[0092] At 134, method 100 includes computing an updated molecular graph based on the updated scalar embeddings and the updated vector embeddings for each node. At 136, an updated molecular graph is computed based on the updated scalar edge embeddings, the updated scalar node embeddings, and the updated vector embeddings.

[0093] At 138, the method 100 includes determining whether the last MPGNN processing block has been completed, if so, the method proceeds to step 140. Otherwise, if not, the method 100 loops back to Fig.15 At step 114 in . At 140, method 100 includes outputting a value of a target molecular property of the molecular system determined based on the updated molecular graph. At 142, the method may include processing the output value for the target molecular property and / or the output updated molecular graph using a downstream program (such as a molecular dynamics program), as described above.

[0094] As described above, the systems and methods described herein have demonstrated technical benefits of improved accuracy and reduced computational cost compared to prior art models.

[0095] In some embodiments, the methods and processes described herein may be bound to a computing system of one or more computing devices. In particular, such methods and processes may be implemented as computer applications or services, application programming interfaces (APIs), libraries, and / or other computer program products.

[0096] Fig.17 A non-limiting embodiment of a computing system 600 that can perform one or more of the above methods and processes is schematically illustrated. The computing system 600 is shown in simplified form. The computing system 600 can embody the above and Figure 1 The computer system 10 shown. The computing system 600 can take the form of one or more personal computers, server computers, tablet computers, home entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smart phones), and / or other computing devices, as well as wearable computing devices (such as smart watches and head-mounted augmented reality devices).

[0097] The computing system 600 includes a logic processor 602, a volatile memory 604, and a non-volatile storage device 606. The computing system 600 may optionally include a display subsystem 608, an input subsystem 610, a communication subsystem 612, and / or Fig.17 Other components not shown.

[0098] Logical processor 602 includes one or more physical devices configured to execute instructions. For example, a logical processor may be configured to execute instructions that are part of one or more applications, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform tasks, implement data types, transform the state of one or more components, implement technical effects, or otherwise achieve desired results.

[0099] The logical processor may include one or more physical processors (hardware) configured to execute software instructions. Additionally or alternatively, the logical processor may include one or more hardware logic circuits or firmware devices configured to execute hardware-implemented logic or firmware instructions. The processor of the logical processor 602 may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel and / or distributed processing. The various components of the logical processor may optionally be distributed in two or more separate devices, which may be remotely located and / or configured for coordinated processing. Various aspects of the logical processor may be virtualized and executed by a remotely accessible networked computing device configured in a cloud computing configuration. In this case, these virtualized aspects are run on different physical logical processors of various different machines, which should be understood.

[0100] The non-volatile storage device 606 includes one or more physical devices configured to hold instructions executable by a logical processor to implement the methods and processes described herein. When implementing such methods and processes, the state of the non-volatile storage device 606 may be transformed, for example, to hold different data.

[0101] The non-volatile storage device 606 may include a removable and / or built-in physical device. The non-volatile storage device 606 may include an optical memory (e.g., CD, DVD, HD-DVD, Blu-ray disc, etc.), a semiconductor memory (e.g., ROM, EPROM, EEPROM, flash memory, etc.), and / or a magnetic memory (e.g., a hard disk drive, a floppy disk drive, a tape drive, MRAM, etc.) or other mass storage device technology. The non-volatile storage device 606 may include a non-volatile, dynamic, static, read / write, read-only, sequential access, location addressable, file addressable, and / or content addressable device. It should be understood that the non-volatile storage device 606 is configured to retain instructions even when the non-volatile storage device 606 is powered off.

[0102] The volatile memory 604 may include a physical device that includes random access memory. The volatile memory 604 is typically used by the logical processor 602 to temporarily store information during the processing of software instructions. It should be understood that when the volatile memory 604 is powered off, the volatile memory 604 typically does not continue to store instructions.

[0103] Aspects of the logic processor 602, volatile memory 604, and non-volatile storage device 606 may be integrated together into one or more hardware logic components. For example, such hardware logic components may include field programmable gate arrays (FPGAs), program-specific and application-specific integrated circuits (PASIC / ASIC), program-specific and application-specific standard products (PSSP / ASSP), systems on chips (SOCs), and complex programmable logic devices (CPLDs).

[0104] The terms "module", "program", and "engine" may be used to describe aspects of a computing system 600 that is typically implemented in software by a processor to use a portion of volatile memory to perform a specific function that involves a transformation process that specifically configures the processor to perform the function. Thus, a module, program, or engine may be instantiated using a portion of volatile memory 604 via a logical processor 602 that executes instructions held by a non-volatile storage device 606. It should be understood that different modules, programs, and / or engines may be instantiated from the same application, service, code block, object, library, routine, API, function, and the like. Likewise, the same module, program, and / or engine may be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, and the like. The terms "module", "program", and "engine" may cover individuals or groups of executable files, data files, libraries, drivers, scripts, database records, and the like.

[0105] When included, the display subsystem 608 can be used to present a visual representation of the data maintained by the non-volatile storage device 606. The visual representation can take the form of a graphical user interface (GUI). Since the methods and processes described herein change the data maintained by the non-volatile storage device, and thus transform the state of the non-volatile storage device, the state of the display subsystem 608 can also be transformed to visually represent the change of the underlying data. The display subsystem 608 can include one or more display devices utilizing almost any type of technology. Such a display device can be combined with the logical processor 602, the volatile memory 604, and / or the non-volatile storage device 606 in a shared housing, or such a display device can be a peripheral display device.

[0106] When included, the input subsystem 610 may include or interface with one or more user input devices such as a keyboard, mouse, touch screen, or game controller. In some embodiments, the input subsystem may include or interface with selected natural user input (NUI) components. Such components may be integrated or peripheral, and the conversion and / or processing of input actions may be processed on-board or off-board. Example NUI components may include microphones for speech and / or speech recognition; infrared, color, stereo, and / or depth cameras for machine vision and / or gesture recognition; head trackers, eye trackers, accelerometers, and / or gyroscopes for motion detection and / or intent recognition; and electric field sensing components for assessing brain activity; and / or any other suitable sensors. When included, the communication subsystem 612 can be configured to communicatively couple the various computing devices described herein to each other and to communicatively couple to other devices. The communication subsystem 612 can include wired and / or wireless communication devices compatible with one or more different communication protocols. As a non-limiting example, the communication subsystem can be configured to communicate via a wireless telephone network or a wired or wireless local area network or wide area network (such as HDMI on a Wi-Fi connection). In some embodiments, the communication subsystem can allow the computing system 600 to send messages to other devices and / or receive messages from other devices via a network such as the Internet.

[0107] The following paragraphs discuss several aspects of the present disclosure. According to one aspect of the present disclosure, a computing system is provided. The system may include one or more processors configured to execute a message passing graph neural network (MPGNN) during an inference phase, the MPGNN including an embedding block, one or more MPGNN processing blocks, and an output block, each MPGNN processing block including a corresponding message passing sub-block and a corresponding update sub-block, wherein the message passing sub-block is configured with a vector scalar interactive message passing mechanism. The processor may also be configured to receive a molecular graph of a molecular system as an input to the MPGNN, the molecular graph including nodes connected by edges, the nodes representing atoms, and the edges representing inter-atomic bonds in the molecular system. The processing may also be configured to process the molecular graph using the embedding block, thereby generating a scalar embedding and a vector embedding, the scalar embedding encoding scalar information describing the characteristics of the nodes and edges, and the vector embedding representing the geometric relationship between the nodes and edges of the molecular graph. The processing may also be configured to process the scalar embedding via the vector scalar interactive message passing mechanism of the message passing sub-block of the MPGNN, thereby generating scalar information, and passing the scalar information from the scalar embedding to the embedding space containing the vector embedding. The processor may also be configured to update the vector embeddings based on an embedding space containing the vector embeddings and the scalar information from the scalar embeddings via the update sub-block. The processor may also be configured to update the scalar embeddings based on a runtime geometric computation of the geometric relationships encoded in the vector embeddings via the update sub-block. The processor may also be configured to calculate an updated molecular graph based on the updated scalar embeddings and the updated vector embeddings for each node via the update sub-block. The processor may also be configured to output, via the output block, a value of a target molecular property of the molecular system determined based on the updated molecular graph.

[0108] According to this aspect, scalar embedding may include scalar node embedding and scalar edge embedding, the scalar node embedding encodes the type of atom represented by each node, and the scalar edge embedding encodes the interatomic distance represented by each edge, and the vector embedding encodes geometric information, which includes a directional unit vector for each node in the molecular graph and a relative position bond angle vector for each node pair in a plurality of node pairs; and processing the molecular graph using an embedding block includes: encoding initial values ​​for the scalar node embedding, scalar edge embedding and vector embedding in the molecular graph via the embedding block.

[0109] According to this aspect, the processor can be configured to process scalar embeddings via a vector-scalar interactive message passing mechanism of a message passing sub-block at least in part by the following operations: for each MPGNN processing block, for each target node among multiple target nodes in the molecular graph, in the MPGNN processing block loop: generate a scalar message via a scalar message function of the message passing sub-block, the scalar message encoding information based on one or more of the following: scalar node embeddings for the target node and adjacent source nodes, scalar edge embeddings for the target node, and calculated attention scores for one or more scalar node embeddings and scalar edge embeddings for the target node from a trained graph attention network; pass the scalar message to the vector message function of the message passing sub-block; and generate a vector message via the vector message function based on the vector embedding and the scalar message.

[0110] According to this aspect, the processor can be configured to generate a scalar message at least in part by the following operations: fusing a scalar node embedding and a scalar edge embedding to thereby generate a fused scalar embedding, wherein the fusion is achieved by concatenation, a Hadamard product, or adding a learnable bias term, and calculating an attention score based on the fused scalar embedding via a non-linear activation function.

[0111] According to this aspect, in the MPGNN processing block loop, the processor can be configured to: before updating the vector embedding and scalar embedding, respectively aggregate the vector embedding and scalar embedding across the source nodes connected to the target node for each target node to generate a corresponding aggregated scalar message and aggregated vector message for the target node.

[0112] According to this aspect, in the MPGNN processing block loop, the processor can be configured to update the vector embedding via the update sub-block at least in part by the following operation: via the update sub-block, updating the vector embedding for the target node based on the aggregated scalar message and the aggregated vector message for the target node.

[0113] According to this aspect, in the MPGNN processing block loop, the processor can be configured to update the scalar embedding via the update sub-block at least in part by the following operations: via the update sub-block, perform runtime geometric calculations to calculate runtime values ​​of the relative position bond angle vector, the direction unit vector and the dihedral angle for each target node; via the update sub-block, calculate the updated scalar node embedding for the target node based on the calculated relative position bond angle vector, the aggregated scalar message for the target node and the scalar node embedding for the target node; and via the update sub-block, calculate the updated scalar edge embedding based on the calculated dihedral angle and the scalar edge embedding.

[0114] According to this aspect, the processor may be configured to compute an updated molecular graph based on the updated scalar edge embeddings, the updated scalar node embeddings, and the updated vector embeddings.

[0115] According to this aspect, the processor may be further configured to: during a training phase prior to the inference phase, train the MPGNN on a training dataset comprising a plurality of molecular graphs for different conformational geometries of a molecular system and corresponding ground truth values ​​of target molecular properties for each molecular graph.

[0116] According to this aspect, the ground truth can be calculated via density functional theory.

[0117] According to this aspect, the molecular property of interest is an energy parameter, a force parameter or a dipole moment.

[0118] According to this aspect, the values ​​for the target molecular properties may be output to a molecular dynamics simulation program for use in molecular dynamics simulation.

[0119] According to another aspect of the present disclosure, a computerized method is provided. The computerized method may include: executing a message passing graph neural network (MPGNN) via one or more processors of a computing device. The computerized method may also include: receiving a molecular graph of a molecular system as an input of the MPGNN, the molecular graph including nodes connected by edges, the nodes representing atoms, and the edges representing interatomic bonds in the molecular system. The computerized method may also include: processing the molecular graph using the MPGNN to generate a scalar embedding and a vector embedding, the scalar embedding encoding scalar information describing the characteristics of the nodes and edges, and the vector embedding representing the geometric relationship between the nodes and edges of the molecular graph. The computerized method may also include: processing the scalar embedding via the vector-scalar interactive message passing mechanism of the MPGNN to generate scalar information, and passing the scalar information from the scalar embedding to an embedding space containing the vector embedding. The computerized method may also include: updating the vector embedding based on the embedding space containing the vector embedding and the scalar information from the scalar embedding. The computerized method may also include: updating the scalar embedding based on the runtime geometric calculation of the geometric relationship encoded in the vector embedding. The computerized method may further include calculating an updated molecular graph based on the updated scalar embedding and the updated vector embedding for each node. The computerized method may further include outputting a value of a target molecular property of the molecular system determined based on the updated molecular graph.

[0120] According to this aspect, scalar embeddings may include scalar node embeddings and scalar edge embeddings, wherein the scalar node embeddings encode the type of atoms represented by each node, and the scalar edge embeddings encode the interatomic distances represented by each edge, and vector embeddings encode geometric information, which includes a directional unit vector for each node in the molecular graph and a relative position bond angle vector for each node pair in a plurality of node pairs; and processing the molecular graph may include: encoding initial values ​​for the scalar node embeddings, scalar edge embeddings, and vector embeddings in the molecular graph.

[0121] According to this aspect, scalar embeddings are processed via a vector-scalar interactive message passing mechanism at least in part by the following operations: for each MPGNN processing block, for each target node among multiple target nodes in a molecular graph, in an MPGNN processing block loop: generating a scalar message via a scalar message function of the MPGNN, the scalar message encoding information based on one or more of the following: scalar node embeddings for the target node and adjacent source nodes, scalar edge embeddings for the target node, and calculated attention scores for one or more scalar node embeddings and scalar edge embeddings for the target node from a trained graph attention network; passing the scalar message to a vector message function of the MPGNN; and generating a vector message via the vector message function based on the vector embedding and the scalar message.

[0122] According to this aspect, generating a scalar message may be accomplished at least in part by: fusing a scalar node embedding and a scalar edge embedding to thereby generate a fused scalar embedding, wherein the fusion may be achieved by concatenation, a Hadamard product, or adding a learnable bias term, and calculating an attention score based on the fused scalar embedding via a nonlinear activation function; and in the MPGNN processing block loop, the computational method may further include: before updating the vector embedding and the scalar embedding, for each target node, respectively aggregating the vector embedding and the scalar embedding across the source nodes connected to the target node to generate a corresponding aggregated scalar message and an aggregated vector message for the target node.

[0123] According to the method, in the MPGNN processing block loop, updating the vector embedding can be completed at least in part by the following operations: updating the vector embedding for the target node based on the aggregated scalar message and the aggregated vector message for the target node; and updating the scalar embedding can be completed at least in part by the following operations: performing runtime geometric calculations to calculate the runtime values ​​of the relative position bond angle vector, the direction unit vector and the dihedral angle for each target node; calculating the updated scalar node embedding for the target node based on the calculated relative position bond angle vector, the aggregated scalar message for the target node and the scalar node embedding for the target node; and calculating the updated scalar edge embedding based on the calculated dihedral angle and scalar edge embedding, wherein the updated molecular graph is calculated based on the updated scalar edge embedding, the updated scalar node embedding and the updated vector embedding.

[0124] According to this aspect, the computerized method may also include: during a training phase prior to the inference phase, training the MPGNN on a training data set, the training data set comprising a plurality of molecular graphs for different conformational geometries of the molecular system, and corresponding ground truth values ​​of target molecular properties for each molecular graph, wherein the target molecular property may be an energy parameter, a force parameter, or a dipole moment.

[0125] According to another aspect of the present disclosure, a computing system is provided. The system may include one or more processors configured to: during the inference phase, execute a message passing graph neural network (MPGNN), the MPGNN including an embedding block, one or more MPGNN processing blocks and an output block, each MPGNN processing block including a corresponding message passing sub-block and a corresponding update sub-block, wherein the message passing sub-block is configured with a vector scalar interactive message passing mechanism. The processor may also be configured to receive a molecular graph of a molecular system as an input to the MPGNN, the molecular graph including nodes connected by edges, the nodes representing atoms, and the edges representing interatomic bonds in the molecular system. The processor may also be configured to process the molecular graph using the embedding block, thereby generating a scalar embedding and a vector embedding, the scalar embedding encoding scalar information describing the characteristics of the nodes and edges, and the vector embedding representing the geometric relationship between the nodes and the edges of the molecular graph. The processor may also be configured to, the processor may also be configured to update the vector embedding based on the vector embedding and the scalar information from the scalar embedding via the update sub-block. The processor may also be configured to update the scalar embedding based on the runtime geometric computation of the geometric relationship encoded in the vector embedding via the update sub-block. The processor may also be configured to calculate an updated molecular graph based on the updated scalar embedding and the updated vector embedding for each node via the update sub-block. The processor may also be configured to output the value of the target molecular property of the molecular system determined based on the updated molecular graph via the output block.

[0126] It should be understood that the configuration and / or method described herein is exemplary in nature, and these specific embodiments or examples should not be considered restrictive, because many variations are possible. The specific routines or methods described herein can represent one or more of any number of processing strategies. Therefore, the various actions shown and / or described can be performed in the order shown and / or described, in other orders, in parallel, or omitted. Likewise, the order of the above process can be changed.

[0127] The subject matter of the present disclosure includes all novel and nonobvious combinations and subcombinations of the various processes, systems and configurations, and other features, functions, acts, and / or properties disclosed herein, as well as any and all equivalents thereof.

Claims

1. A computing system comprising: One or more processors configured to: During the inference phase, executing a message passing graph neural network (MPGNN), the message passing graph neural network comprising an embedding block, one or more MPGNN processing blocks, and an output block, each MPGNN processing block comprising a corresponding message passing sub-block and a corresponding update sub-block, wherein the message passing sub-blocks are configured with a vector-scalar interactive message passing mechanism; receiving a molecular graph of a molecular system as an input to the MPGNN, the molecular graph comprising nodes connected by edges, the nodes representing atoms, and the edges representing bonds between atoms in the molecular system; Processing the molecular graph using the embedding block to generate a scalar embedding and a vector embedding, the scalar embedding encoding scalar information describing characteristics of the nodes and the edges, the vector embedding representing geometric relationships between the nodes and the edges of the molecular graph; processing the scalar embedding via the vector-scalar interactive message passing mechanism of the message passing sub-block of the MPGNN to generate the scalar information, and passing the scalar information from the scalar embedding to an embedding space containing the vector embedding; updating, via the updating sub-block, the vector embedding based on the embedding space containing the vector embedding and the scalar information from the scalar embedding; updating, via the update sub-block, the scalar embedding based on a runtime geometric computation of the geometric relationship encoded in the vector embedding; calculating, via the updating sub-block, an updated molecular graph based on the updated scalar embedding and the updated vector embedding for each node; as well as The value of the target molecular property of the molecular system determined based on the updated molecular graph is output via the output block.

2. The computing system of claim 1, wherein The scalar embeddings include scalar node embeddings and scalar edge embeddings, the scalar node embeddings encode the type of atoms represented by each node, and the scalar edge embeddings encode the interatomic distances represented by each edge, and the vector embeddings encode geometric information, the geometric information including a directional unit vector for each node in the molecular graph and a relative position bond angle vector for each node pair in a plurality of node pairs; and Processing the molecular graph using the embedding block includes: Via the embedding block, initial values ​​for the scalar node embeddings, the scalar edge embeddings, and the vector embeddings in the molecular graph are encoded.

3. The computing system of claim 2, wherein The processor is configured to process the scalar embedding via the vector-scalar interactive message passing mechanism of the message passing sub-block at least in part by: For each MPGNN processing block, for each target node in the plurality of target nodes in the molecular graph, in the MPGNN processing block loop: generating, via a scalar message function of the message passing sub-block, a scalar message encoding information based on one or more of: the scalar node embeddings for the target node and adjacent source nodes, the scalar edge embeddings for the target node, and a calculated attention score for one or more of the scalar node embeddings and the scalar edge embeddings for the target node from a trained graph attention network; Passing the scalar message to the vector message function of the message passing sub-block; as well as A vector message is generated based on the vector embedding and the scalar message via the vector message function.

4. The computing system of claim 3, wherein the processor is configured to generate the scalar message at least in part by: The scalar node embedding and the scalar edge embedding are fused to thereby generate a fused scalar embedding, the fusion being achieved by concatenation, a Hadamard product, or adding a learnable bias term, and the attention score is calculated based on the fused scalar embedding via a non-linear activation function.

5. The computing system of claim 3, wherein: In the MPGNN processing block loop, the processor is configured to: Prior to updating the vector embeddings and the scalar embeddings, the vector embeddings and the scalar embeddings across source nodes connected to the target node are respectively aggregated for each target node to generate a corresponding aggregated scalar message and an aggregated vector message for the target node.

6. The computing system of claim 5, wherein: In the MPGNN processing block loop, the processor is configured to update the vector embedding via the update sub-block at least in part by: The vector embedding for the target node is updated based on the aggregated scalar message and the aggregated vector message for the target node via the update sub-block.

7. The computing system of claim 6, wherein: In the MPGNN processing block loop, the processor is configured to update the scalar embedding via the update sub-block at least in part by: performing the runtime geometric calculation via the update sub-block to calculate runtime values ​​of the relative position bond angle vector, the direction unit vector and the dihedral angle for each target node; calculating, via the update sub-block, an updated scalar node embedding for the target node based on the calculated relative position bond angle vector, the aggregated scalar message for the target node, and the scalar node embedding for the target node; as well as An updated scalar edge embedding is calculated via the updating sub-block based on the calculated dihedral angle and the scalar edge embedding.

8. The computing system of claim 7, wherein the processor is configured to compute an updated molecular graph based on the updated scalar edge embeddings, the updated scalar node embeddings, and the updated vector embeddings.

9. The computing system of claim 1 , wherein the processor is further configured to: During the training phase preceding the inference phase, The MPGNN is trained on a training dataset comprising a plurality of molecular graphs for different conformational geometries of the molecular system and a corresponding ground truth value of the target molecular property for each molecular graph.

10. The computing system of claim 9, wherein the ground truth is calculated via density functional theory.

11. The computing system of claim 1, wherein the target molecular property is an energy parameter, a force parameter, or a dipole moment.

12. The computing system of claim 1, wherein the value for the target molecular property is output to a molecular dynamics simulation program for use in a molecular dynamics simulation.

13. A computerized method comprising: Executing, via one or more processors of a computing device, a message passing graph neural network (MPGNN): receiving a molecular graph of a molecular system as an input to the MPGNN, the molecular graph comprising nodes connected by edges, the nodes representing atoms, and the edges representing bonds between atoms in the molecular system; Processing the molecular graph using the MPGNN to generate a scalar embedding and a vector embedding, wherein the scalar embedding encodes scalar information describing features of the nodes and the edges, and the vector embedding represents a geometric relationship between the nodes and the edges of the molecular graph; Processing the scalar embedding via a vector-scalar interactive message passing mechanism of the MPGNN to generate the scalar information, and passing the scalar information from the scalar embedding to an embedding space containing the vector embedding; updating the vector embedding based on the embedding space including the vector embedding and scalar information from the scalar embedding; updating the scalar embedding based on a runtime geometric computation of the geometric relationship encoded in the vector embedding; computing an updated molecular graph based on the updated scalar embedding and the updated vector embedding for each node; and The value of the target molecular property of the molecular system determined based on the updated molecular graph is output.

14. The computerized method of claim 13, wherein The scalar embeddings include scalar node embeddings and scalar edge embeddings, the scalar node embeddings encode the type of atoms represented by each node, and the scalar edge embeddings encode the interatomic distances represented by each edge, and the vector embeddings encode geometric information, the geometric information including a directional unit vector for each node in the molecular graph and a relative position bond angle vector for each node pair in a plurality of node pairs; and Processing the molecular graph comprises: Initial values ​​for the scalar node embeddings, the scalar edge embeddings, and the vector embeddings in the molecular graph are encoded.

15. The computerized method of claim 14, wherein Processing the scalar embedding via the vector-scalar interactive message passing mechanism at least in part by: For each MPGNN processing block, for each target node in the plurality of target nodes in the molecular graph, in the MPGNN processing block loop: generating, via a scalar message function of the MPGNN, a scalar message encoding information based on one or more of: the scalar node embeddings for the target node and adjacent source nodes, the scalar edge embeddings for the target node, and a computed attention score for one or more of the scalar node embeddings and the scalar edge embeddings for the target node from a trained graph attention network; Passing the scalar message to the vector message function of the MPGNN; as well as A vector message is generated based on the vector embedding and the scalar message via the vector message function.

Images