Solubility prediction method for coupling directional message passing neural network and hybrid expert model
By combining the solubility prediction methods of DMPNN and MoE modules, the solubility prediction problems of various solute-solvent systems and different temperatures are solved, and high-precision and stable prediction effects are achieved, improving the adaptability and robustness of the model.
Patent Information
- Application Number
- CN202510481533.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-16
- Publication Date
- 2025-08-01
AI Technical Summary
The existing solubility prediction methods have limited prediction accuracy and generalization ability in the face of multiple solute-solvent systems and different temperature conditions, and have failed to fully consider the changes in solubility with temperature and the impact of different solute-solvent systems.
The method of coupled directional message delivery neural network (DMPNN) and hybrid expert (MoE) module is adopted to improve prediction of solubility through molecular graph representation and feature extraction, combined with temperature information, and capture molecular structure information by using DMPNN, and the MoE module performs dynamic weight allocation to improve prediction accuracy and generalization capabilities.
High-precision and stable solubility prediction in multiple solute-solvent systems and different temperatures are achieved, which improves the adaptability and robustness of the model and significantly improves the prediction accuracy and generalization ability.
Smart Images

Figure CN120412799A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of chemoinformatics and artificial intelligence, and particularly to a method for predicting molecular solubility using deep learning technology. Background Art
[0002] Solubility, as a key physicochemical index in chemistry, pharmacy, and materials science, is widely used in fields such as drug screening and new material design. Traditional solubility prediction methods mainly rely on empirical models or classical physicochemical formulas, suffering from problems such as high computational complexity and limited prediction effectiveness. In recent years, the development of deep learning technology has provided new ideas for solubility prediction, and several deep learning-based solubility prediction methods have emerged.
[0003] For example, patent document CN109741797B discloses a method for predicting the water solubility level of small molecule compounds using deep learning technology. This method inputs compound structure information (such as SMILES text), performs data cleaning, feature extraction, and normalization processing, constructs a model for training, and thus realizes the prediction of the solubility of compounds in water (under normal temperature conditions); while patent document CN114093435B discloses a method for predicting the water solubility of chemical molecules based on a bidirectional time series prediction model and an attention mechanism, which predicts by learning the correspondence between the chemical molecule structure sequence and water solubility. Both of these two technologies mainly focus on solubility prediction at normal temperature with water as the solvent, and their application conditions are relatively fixed.
[0004] In addition, patent document CN114334022B provides a transfer learning model for predicting the solubilization effect of cyclodextrin on compound molecules. This model first constructs a pre-trained model by pre-training the SMILES features of a large number of compounds, then migrates the data to the model application domain adaptation fine-tuning stage and the QSPR modeling stage, and finally uses the cyclodextrin-included compounds and corresponding solubility data for learning to achieve the prediction of the solubilization effect of cyclodextrin-included compounds. This method solves the problem of low accuracy caused by insufficient solubility data of cyclodextrin inclusion compounds and is applicable to various cyclodextrin inclusion compounds, showing strong generalization ability, but its technical focus is concentrated on the cyclodextrin inclusion system.
[0005] None of the above technologies fully consider the influence of solubility changes with temperature and different solute-solvent systems, and in practical applications, predictions often need to be made in a changing environment. Summary of the Invention
[0006] To address the problems raised in the background art, the present invention proposes a solubility prediction method that couples a directed message passing neural network (DMPNN) and a mixture of experts (MoE) module. Compared with the aforementioned technologies, the present invention not only predicts the solubility in a single solvent (such as water) or at a fixed temperature, but extends the prediction to any temperature and multiple solute-solvent systems, thus adapting to a wider range of application scenarios.
[0007] The present invention relates to a solubility prediction method based on deep learning, in particular, a solubility prediction method that couples dual DMPNN and MoE modules to accurately predict the solubility of solutes under multiple solute-solvent systems and different temperature conditions.
[0008] Traditional solubility prediction methods mostly rely on empirical formulas or simple statistical learning methods, which usually can only handle limited solvents and temperature conditions, and have limited accuracy and generalization ability when facing multiple solute-solvent systems and different temperatures. To overcome the deficiencies of existing methods, the present invention proposes a deep learning model that couples molecular graph representation and expert networks, which can efficiently and accurately predict solubility under multiple solute-solvent systems and different temperatures.
[0009] The main technical solutions of the present invention are as follows:
[0010] (1) Molecular graph representation of solutes and solvents: First, the molecular structures of solutes and solvents are respectively represented as graph structures, where nodes represent atoms (encoding atomic types, valence electron numbers, hydrogen bond donor and acceptor characteristics, etc., as shown in Table 1), and edges represent chemical bonds (encoding bond types, conjugation, and stereochemical characteristics, as shown in Table 2), and their feature encoding is performed. Through this graph structure representation method, local and global structural information between molecules can be better captured as the input of the deep learning model. More preferably, the node features include at least one of atomic type, valence electron number, formal charge, number of hydrogen atoms, hybridization state, number of lone pairs of electrons, hydrogen bond donor, hydrogen bond acceptor, label in atomic ring, electronegativity, aromaticity, atomic mass, Gasteiger Charges, and Crippen Contribs; the edge features include at least one of whether it is a single bond, whether it is a double bond, whether it is a triple bond, whether it is an aromatic bond, whether it is conjugated, whether it is in a ring, and stereochemistry.
[0011] (2) Dual DMPNN Feature Extraction: The dual DMPNN model is used to perform message passing and feature extraction on the molecular graphs of solutes and solvents respectively, and the graph neural network is utilized to capture the key information of the molecular structure; More preferably, specifically for the molecular graphs of solutes and solvents, two independent DMPNNs are used for feature extraction respectively. Each DMPNN performs message passing according to the graph structure, thereby gradually updating the features of each node, and finally obtaining the molecular representations of solutes and solvents. Compared with traditional molecular descriptor methods, DMPNN can make more full use of the deep information in the molecular structure. Even more preferably, the DMPNN updates features through the following steps:
[0012] a) Initialize the hidden layer features of the directed edges. After concatenating the node features and edge features, they are fused through a learnable weight matrix to obtain the initial edge hidden state;
[0013] b) In the message passing stage, for each directed edge, in multiple iterations, the message is accumulated according to the hidden states of other edges pointing to the same node except the reverse edge of the target edge, and then it is updated by combining the initial edge state and the weight matrix;
[0014] c) In the aggregation stage, the sum of the incoming edge hidden states of each node is calculated to generate the node information representation, and then the final node representation is calculated using the learnable weight matrix and activation function. Finally, the representations of all nodes are aggregated to obtain the molecular graph-level representation.
[0015] Even more preferably, the iterative update in step b) includes: For each directed edge v→w, its message passing follows the following rule: First, calculate the cumulative sum of the hidden layer features of all incoming edges (k→v, k≠w) pointing to the current starting node v except w→v to generate the message to be passed; Subsequently, the message is linearly transformed through the learnable weight matrix, and the hidden layer features of the directed edge v→w are updated by combining the initial edge features. This directional message passing strategy effectively avoids the redundancy and noise caused by the message passing back and forth, thereby more accurately capturing the local structure information inside the molecule. Specifically, in the embodiment, the message passed to the directed edge v→w is calculated by formula 2, where represents the cumulative sum of the hidden layer features of any directed edge k→v (k≠w) pointing to node v except the reverse edge w→v, and the hidden layer features of the directed edge v→w are updated by formula 3.
[0016] (3) Feature Fusion: The obtained molecular feature vectors of solutes and solvents are fused with environmental information including but not limited to temperature and other molecular descriptors to form a unified input representation; More preferably, specifically, the feature vectors obtained from the DMPNNs of solutes and solvents are fused with external temperature information and molecular descriptors (such as molecular weight, heavy atom weight, SlogP _VSA, TPSA, MolLogP, etc. (as shown in Table 3) are cascaded to form a unified input representation. In this way, the model can not only consider the structural characteristics of the molecule itself, but also incorporate environmental factors (such as temperature) and traditional molecular property information into the prediction model.
[0017] (4) MoE module: The fused feature input representation is input into the MoE module. The MoE module includes multiple parallel expert networks and a gating network. The gating network dynamically assigns weights to each expert based on the input features and performs weighted fusion on the outputs of the expert networks. Each expert network independently learns different tasks, and the gating network automatically selects the most suitable expert for weighted output according to the requirements of the task, thereby improving the prediction accuracy and generalization ability of the model. Further preferably, the gating network dynamically generates the weights of each expert according to the input features (Formula 9), and normalizes the weights of the expert networks through the Softmax function.
[0018] (5) Solubility prediction: Finally, the weighted output processed by the MoE module is used for solubility prediction. This method can adapt to a variety of solvent-solute systems and exhibits high accuracy and stable prediction ability at different temperatures. Further preferably, the solubility prediction method adopts a cross-validation strategy for training and evaluation to ensure the stability and robustness of the model on different data subsets. The cross-validation strategy is 10-fold cross-validation, and when dividing the dataset, it is ensured that the samples of the same solute-solvent pair at different temperatures do not cross-fold distribution.
[0019] For the above-mentioned technical solution, further preferably, the method uses molecular graph data to represent the molecular structures of the solute and the solvent. The molecular graph includes node and edge features. For example, the node features may include atomic type, number of valence electrons, number of hydrogen atoms, etc., and the edge features may include chemical bond type, whether conjugated, and stereochemical characteristics, etc.
[0020] For the above-mentioned technical solution, further preferably, the feature representation of the molecular graph data is updated through the iterative update of the directed edge features of DMPNN (Formulas 1 - 3), so as to more accurately capture the chemical characteristics in the molecular structure (extract directional molecular features).
[0021] For the above-mentioned technical solution, further preferably, the updated molecular graph features are fused with temperature information and molecular descriptors (such as molecular weight, heavy atom weight, topological polar surface area) to form a comprehensive input representation.
[0022] For the technical solutions described above, more preferably, the comprehensive input representation is passed into the MoE module, which dynamically generates the weights of each expert network according to the input data through a gating network, and performs weighted fusion on the outputs of each expert network, thereby generating the prediction result of solubility. Even more preferably, the expert networks of the MoE module independently process different subspaces of the input features, and the gating network normalizes the weights using the Softmax function.
[0023] For the technical solutions described above, more preferably, the method is applicable to the solubility prediction of multiple solute-solvent systems and under different temperature conditions, and can achieve better prediction performance than traditional methods on large-scale data sets.
[0024] For the technical solutions described above, more preferably, the solubility prediction method adopts a cross-validation strategy for training and evaluation to ensure the stability and robustness of the model on different data subsets.
[0025] Another aspect of the present invention is to protect a computer-readable storage medium for implementing the method described above. A computer program is stored in the storage medium, and when the program is executed on a computer, it can implement the above-mentioned solubility prediction method based on the dual DMPNN and MoE modules. More preferably, the computer-readable storage medium stores a program for executing the above method, and the program includes the implementation codes of the dual DMPNN and MoE modules based on the PyTorch framework.
[0026] Another aspect of the present invention is to protect a system for implementing the method described above. The system includes:
[0027] (1) A data input module for receiving the molecular data of solute and solvent, temperature information, and molecular descriptors;
[0028] (2) A feature extraction module for converting the molecular structures (molecular data of solute and solvent) into molecular graphs with node and edge features;
[0029] (3) A dual DMPNN module for respectively performing directional feature extraction on the solute and solvent molecular graphs, including respectively performing message passing and feature fusion on the solute and solvent molecular graphs; the dual DMPNN module performs directional feature extraction through iterative update of the directed edge features, including initializing the directed edge hidden layer according to the node features and edge features, iteratively updating the message passing, and generating the molecular graph-level representation;
[0030] (4) MoE module, which fuses the outputs of the expert networks through a dynamic gating network (Equations 9 - 10). Further preferably, this module includes multiple parallel expert networks and a gating network. The gating network dynamically generates the weights of each expert network based on the comprehensive input, and weighted fuses their outputs to obtain the solubility prediction result;
[0031] (5) Output module, which is used to generate the final solubility prediction value.
[0032] For the above - mentioned technical solution, further preferably, in the hybrid expert module, each expert network is composed of a multi - layer fully - connected neural network and each focuses on extracting information of different dimensions in the input features to capture diverse feature expressions. Through the differential processing of the input features by the parallel expert networks, the model can adaptively capture the complex non - linear relationship between solute - solvent interactions and temperature conditions.
[0033] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0034] The present invention introduces a hybrid of a direction - aware message - passing neural network (DMPNN) and a dynamically - gated mixture of experts (MoE) module:
[0035] On the one hand, based on the molecular graph feature extraction mechanism of the dual DMPNN, through the iterative update of the directed - edge features (see Equations 1 - 3), it not only fuses the information of solutes and solvents at the atomic and bond levels but also can capture the topological structure features inside the molecule. This architecture uses the directed - edge message - passing design, effectively avoiding information redundancy and noise interference, and thus showing higher stability and robustness in structural information extraction, making up for the deficiencies of traditional MPNN in this regard.
[0036] On the other hand, through the dynamic weight allocation of the MoE module (Equations 8 - 10), combined with the multi - dimensional fusion of temperature and molecular descriptors (Equation 7), it realizes the differential collaborative prediction of expert networks under different solute - solvent systems and temperature conditions, breaking through the expression bottleneck of traditional feed - forward neural networks for complex solubility laws.
[0037] Experimental data show (Tables 5 - 7) that in the 10 - fold cross - validation covering 56,945 multi - solvent and wide - temperature - range datasets, the MSE of this method reaches 0.1815±0.0256, a 6.9% reduction compared to the optimal comparison model (DMPNN), and still maintains R 2 = 0.7663±0.0083 in the external small - sample test set (2380 low - frequency solvent data), verifying its strong adaptability to multi - solvent systems, temperature dependence, and long - tail distributions; ablation experiments ( Figure 4, Table 6) further confirmed that the directional feature extraction of DMPNN makes the prediction error distribution more concentrated (MAE reduced by 3.0%), while the MoE module significantly improves the model robustness through the dynamic weight mechanism (the standard deviation of errors reduced by 48.3%). The synergistic effect of the two finally realizes the solubility prediction ability with high precision and high generalization, providing a reliable tool for the analysis of complex dissolution behaviors in drug development and material design.
[0038] The solubility prediction method of the present invention significantly improves the accuracy and generalization ability of solubility prediction through deep learning, combined with molecular graph representation, dual DMPNN feature extraction, and MoE module. Compared with the prior art, the present invention can handle more complex solubility prediction tasks, and has stronger adaptability and robustness, with high practical application value. Brief Description of the Drawings
[0039] Figure 1 It is a schematic structural diagram of the graph neural network message passing model of the molecule.
[0040] Figure 2 It is a schematic structural diagram of the MoE model.
[0041] Figure 3 It is a prediction performance graph of 10-fold cross-validation in the embodiment of the present invention.
[0042] Figure 4 It is a violin plot of the total error distribution of different models in the ablation experiment in the embodiment of the present invention on 10 folds. Detailed Embodiments
[0043] As shown in the attached Figure 1 and Figure 2 , the present system mainly consists of the following parts:
[0044] Input module: including the molecular graphs of solute and solvent, temperature information, and other molecular descriptors (such as molecular weight, heavy atom weight, SlogP_VSA, TPSA, MolLogP).
[0045] Dual DMPNN feature extraction module: respectively perform feature extraction on the molecular graphs of solute and solvent.
[0046] Feature fusion module: perform concatenation processing on the features extracted by dual DMPNN with temperature and other descriptors.
[0047] MoE module: including multiple parallel expert networks and gating networks, dynamically generating expert weights according to the input features to achieve collaborative prediction of expert networks.
[0048] Output module: perform regression processing on the output of the MoE module to finally obtain the solubility prediction result.
[0049] Gasteiger Charges and Crippen Contribs: The specific calculation method references the ComputeGasteigerCharges function of RDKit. Test set partitioning method: Count the frequencies by solvent name and select solvent samples with frequencies < 100.
[0050] The technical solution of the present invention will be further described in detail below in conjunction with specific embodiments. It should be noted that the specific features (such as node and edge features, etc.) or hyperparameters (such as the number of network layers, hidden layer dimensions, activation functions, etc.) in each module can be adjusted and optimized according to the actual application scenario. This embodiment is only a specific application example, and other equivalent solutions are within the protection scope of the present invention. The specific steps are as follows:
[0051] Step 1, Preparation of input data: First, using the SMILES notation, extract the molecular structures of solutes and solvents through RDKit 2022.09.5, perform feature encoding, and convert the molecules into a graph data structure, where nodes represent atoms (such as atom type, hybridization state, atomic mass, electronegativity, etc.), and edges represent chemical bonds (such as bond type, bond length, etc.); at the same time, record the temperature information. See the examples and descriptions of node and edge features in Tables 1 and 2 for details.
[0052] Table 1. Node Feature Description
[0053]
[0054]
[0055] Table 2. Edge Feature Description
[0056]
[0057] Step 2, Dual DMPNN Feature Extraction Module: Independently construct DMPNNs for solutes and solvents respectively to achieve efficient extraction of molecular structure information.
[0058] Step 2.1, Initialization: First, the model performs a fusion process on the features of nodes and edges through Equation 1. Among them, represents the initialized hidden layer feature of the directed edge v→w, τ is the activation function, x v is the initial feature of node v, e vw is the feature of the directed edge v→w, and cat represents concatenation. W i represents the learnable weight matrix.
[0059]
[0060] Step 2.2, Message Passing: In the message passing phase, through multiple iterative updates, the hidden layer features of each directed edge v→w will be updated. For the message passed to the edge v→w, the calculation method is Formula 2, where, denotes the message passed to the directed edge v→w, denotes the accumulation of the hidden layer features of any directed edge k→v (except the directed edge w→v) pointing to the node v. The update formula for the edge v→w is Formula 3, where, denotes the hidden layer of the directed edge v→w obtained after update, and W m denotes the weight matrix. This process will be iteratively updated for t∈{1, 2,..., T}.
[0061]
[0062] Step 2.3, Graph-Level Representation: First, sum the information from neighboring nodes to generate the information representation m v of the node v, as shown in Formula 4. Subsequently, the final representation h v of the node is calculated by Formula 5, where W a denotes the learnable weight matrix. In our implementation, the readout function obtains the feature vector of the entire molecule by summing the hidden states of all nodes, as shown in Formula 6.
[0063]
[0064] h v = τ(W a ·cat(x v , m v ))#(5)
[0065]
[0066] Step 3, Feature Fusion Module: Concatenate the features obtained by the dual DMPNN with other auxiliary information, as shown in Formula 7, where T is the temperature and x m is a vector containing information such as molecular weight and heavy atom weight. Table 3 shows the examples and descriptions of the x m used. This step enables the model to utilize both molecular structure information and traditional descriptors to capture more comprehensive feature information.
[0067] h = cat(h m , T, x m )#(7)
[0068] Table 3. Molecular-Level Descriptor Descriptions
[0069] Category Specific description Molecular weight property MolWt, HeavyAtomMolWt Molecular surface hydrophobicity distribution SlogP_VSA1 - SlogP_VSA12 Polar surface area TPSA Lipophilicity index MolLogP
[0070] Step 4, MoE module: Take the fused feature h as the input, and further extract and integrate information through the MoE module to achieve more refined prediction.
[0071] Step 4.1, Expert networks: Set up several parallel expert networks. Each expert network consists of multiple layers of fully connected layers and independently processes different subspaces of the input features, so as to extract various patterns in the input features, as shown in Equation 8.
[0072]
[0073] Step 4.2, Gating network: Output a weight vector based on the fused feature h to assign weights to each expert network. Usually, the softmax function is used to ensure that the sum of the weights is 1, as shown in Equation 9, where W g is the weight matrix of the gating network.
[0074] g(h) = softmax(W g ·h) #(9)
[0075] Step 4.3, Output fusion: The final output y is the weighted sum of the outputs of all expert networks. The weights g i (h) assigned by the gating network to each expert network are used to perform weighted combination on the outputs of the experts to obtain the final prediction result, as shown in Equation 10.
[0076]
[0077] Step 5, Solubility Prediction Model Training Settings: Use the mean squared error (MSE) as the loss function, as shown in Equation 11. Use the Adam optimizer to train the entire network and update all weights including the double DMPNN and MoE module. Adopt a 10-fold cross-validation strategy, divide the dataset into 10 non-overlapping subsets in units of "solute-solvent pairs", ensuring that samples of the same solute-solvent pair at different temperatures do not appear in the training set and validation set simultaneously to improve the requirements for testing the generalization ability of the model. During the training process, the temperature and other molecular descriptors are standardized. The initial learning rate is set to 0.001, and a dynamic adjustment mechanism is designed. When the validation set loss does not decrease significantly for 5 consecutive epochs, the learning rate is reduced to one-tenth of the current value; if there is no improvement for 10 consecutive epochs, the early stopping mechanism is triggered. At the same time, in the MoE module, the number of experts is selected as 4, and there are 2 hidden layers inside each expert module. A Dropout of 0.05 is applied to the fully connected layer of the expert layer to prevent overfitting; in addition, the bias term of the message passing process is not enabled during the training process to avoid interference with information propagation. All models are implemented based on PyTorch 2.2.1, and PyTorch Lightning 2.2 is used for training and experiment management. Table 4 shows an example of the settings of some key hyperparameters. It should be noted that the specific hyperparameters in the above modules (such as the number of network layers, the dimension of the hidden layer, the type of activation function, etc.) can be adjusted and optimized according to the specific application scenario and data characteristics. This embodiment is only a specific application method, and other equivalent technical solutions are within the protection scope of the present invention.
[0078]
[0079] Table 4. Example of Key Hyperparameter Settings
[0080] Parameter item Set value Hidden layer dimension 400 Number of message passing cycles 4 Activation function Leaky ReLU Number of experts 4 Number of hidden layers of experts 2 Initial learning rate 0.001 Dropout 0.05 Number of cross - validation folds 10 - fold Optimizer Adam
[0081] Step 6, Experimental Verification and Performance Evaluation: The dataset sources of this study include the public database BigSolDB, CombiSolu-Exp, and experimental solubility data collected from the literature. Use the van't hoff solubility empirical equation to fit the dataset. The equation is shown in Equation 12, where x is the mole fraction solubility, A and B are empirical parameters, and T is the temperature. Use the fitted R 2 as the selection criterion, and delete the solute-solvent pair data with an R 2 value less than 0.95. Finally, 56,945 high-quality solubility data are selected for model training and verification. Table 5 shows the results of 10-fold cross-validation, including the statistical data of the mean squared error (MSE), root mean squared error (RMSE), coefficient of determination (R 2 ) and mean absolute error (MAE) for each fold. At the same time, attachFigure 3 A scatter plot comparing the predicted values of the validation set of the 10-fold model with the experimental logS values is also provided. Each data point is colored according to its local density, and the darker the color, the higher the frequency. Most of the data points are closely clustered along the diagonal, demonstrating that the developed DMPNN-MoE model has strong prediction accuracy and generalization ability. On average, the MSE of this model is 0.1815 ± 0.0256, R 2 is 0.8634 ± 0.0165, and the MAE is 0.2557 ± 0.0096. The experimental results show that this method achieves low error metrics under various solute-solvent systems and different temperature conditions, and has good prediction accuracy and stable generalization ability, fully verifying the effectiveness and practicality of the present invention.
[0082]
[0083] Table 5. Performance display results of 10-fold cross-validation of the DMPNN-MoE model in this example
[0084] Index MSE RMSE <![CDATA[R 2 > MAE Fold 1 0.1610 0.4013 0.8715 0.2393 Fold 2 0.1808 0.4252 0.8708 0.2565 Fold 3 0.2329 0.4826 0.8312 0.2733 Fold 4 0.1961 0.4428 0.8634 0.2613 Fold 5 0.1315 0.3627 0.8915 0.2443 Fold 6 0.1799 0.4242 0.8672 0.2596 Fold 7 0.1939 0.4403 0.8501 0.2599 Fold 8 0.1710 0.4135 0.8724 0.2449 Fold 9 0.2000 0.4473 0.8432 0.2597 Fold 10 0.1679 0.4097 0.8729 0.2583 Average value 0.1815 0.4249 0.8634 0.2557 Standard deviation 0.0256 0.0303 0.0165 0.0096
[0085] To better demonstrate the competitiveness of the DMPNN-MoE model of the present invention, ablation comparison experiments were conducted in this example: compared models based on graph representation (including DMPNN, MPNN-MoE, and MPNN), and models combining fixed molecular descriptors (RDKit or Mol2vec) with traditional machine learning algorithms, including artificial neural network (ANN), MoE, random forest (RF), and support vector regression (SVR). The latter uses all 213 descriptors of RDKit or fragments with a radius of 1 generated based on the Morgan algorithm (300-dimensional molecular features obtained through skip-gram pre-training). The hyperparameters of the ANN and MoE models are set similarly to the graph-based models, but if there is no obvious improvement in the loss within 5 consecutive evaluation cycles on the validation set, the learning rate is reduced to 0.5 times the current value, while the traditional machine learning models RF and SVR use default hyperparameters (RF is configured with 500 trees; SVR uses an RBF kernel, C = 1.0, gamma is set to "scale", and epsilon = 0.1).
[0086] Table 6 clearly shows the performance metrics of various models in the ablation experiment. Among these models, the DMPNN-MoE model has the highest performance, with MSE = 0.1815 ± 0.0256, R 2= 0.8634 ± 0.0165, MAE = 0.2557 ± 0.0096. This indicates that the integration of DMPNN and the MoE module has significant advantages in the prediction task. In the ablation experiment, the comparison between DMPNN-MoE and MPNN-MoE, as well as between DMPNN and MPNN, shows that adding direction information to the MPNN model is beneficial for improving prediction accuracy. In addition, the comparison between the models with the MoE module and the models without the MoE module (i.e., DMPNN-MoE vs. DMPNN and MPNN-MoE vs. MPNN) confirms that the MoE module shows strong adaptability in different samples, further improving the model performance. In summary, the results show that graph-based deep learning methods (such as models based on DMPNN and MPNN) tend to outperform traditional machine learning methods that rely on fixed descriptors (such as Mol2vec and RDKit descriptors).
[0087] Table 6. Performance of different models in the ablation experiment
[0088] Model MSE <![CDATA[R 2 > MAE DMPNN - MoE 0.1815±0.0256 0.8634±0.0165 0.2557±0.0096 DMPNN 0.1950±0.0283 0.8534±0.0176 0.2637±0.0197 MPNN - MoE 0.2030±0.0300 0.8475±0.0175 0.2713±0.0145 MPNN 0.2058±0.0428 0.8456±0.0259 0.2767±0.0283 RDkit - MoE 0.2281±0.0301 0.8286±0.0167 0.2972±0.0200 RDkit - ANN 0.2406±0.0688 0.8201±0.0430 0.2924±0.0192 Mol2vec - MoE 0.2656±0.0573 0.8008±0.0345 0.3223±0.0145 Mol2vec - ANN 0.2695±0.0885 0.7986±0.0563 0.3227±0.0205 RDkit - RF 0.2517±0.0287 0.8103±0.0193 0.3093±0.0144 RDkit - SVR 0.2908±0.0298 0.7812±0.0149 0.3445±0.0118 Mol2vec - RF 0.2792±0.0314 0.7893±0.0225 0.3384±0.0154 Mol2vec - SVR 0.3746±0.0352 0.7178±0.0209 0.4122±0.0175
[0089] To more intuitively show the error distribution of each index of each model in the ablation experiment during 10-fold cross-validation, this embodiment also uses a violin plot (see Appendix Figure 4 ) to visualize the error distribution characteristics of the first 8 models in Table 6. The violin plot further shows that the error distribution of the DMPNN-MoE solubility model is the most compact and symmetric, reflecting its excellent robustness and generalization ability. The distribution of the model that only uses the feed-forward layer without the MoE module is wider and has a larger deviation, and the feature extraction ability of DMPNN is better than that of MPNN and the other two descriptors.
[0090] To demonstrate the generalization ability of the DMPNN-MoE solubility model of the present invention under small-sample data, this embodiment divides the dataset into two parts: one is the training-validation set (a total of 54,565 solute-solvent pairs), where the number of samples of each solvent is not less than 100; the other is the external test set (a total of 2,380 solute-solvent pairs), where the sample frequency of each solvent is less than 100. Through this division method, the prediction effect of the model on solvents with small samples and long-tailed distributions can be evaluated separately without affecting the model's learning of common solvents. In the training-validation set part, 10-fold cross-validation training is performed using the method in the above steps. Subsequently, the trained model is used to make predictions on the external test set, and the evaluation metrics (MSE, RMSE, R 2, MAE). Table 7 lists the detailed metrics obtained by the external test set under the corresponding models of each fold of cross-validation. It can be seen that in the scenario of small samples and rare solvents, the model still maintains good prediction accuracy and robustness.
[0091] Table 7. Performance of the 10-fold model in the external small-sample test set
[0092] Index MSE RMSE <![CDATA[R 2 > MAE Fold 1 0.2708 0.5204 0.7606 0.3784 Fold 2 0.2644 0.5142 0.7662 0.3632 Fold 3 0.2630 0.5129 0.7675 0.3655 Fold 4 0.2702 0.5198 0.7611 0.3639 Fold 5 0.2740 0.5234 0.7578 0.3749 Fold 6 0.2620 0.5119 0.7684 0.3707 Fold 7 0.2393 0.4891 0.7885 0.3569 Fold 8 0.2705 0.5201 0.7608 0.3607 Fold 9 0.2691 0.5187 0.7621 0.3624 Fold 10 0.2608 0.5107 0.7694 0.3531 Average value 0.2644 0.5141 0.7663 0.3650 Standard deviation 0.0094 0.0093 0.0083 0.0074
[0093] In summary, the present invention proposes a solubility prediction method that couples the DMPNN and MoE modules. The experimental results show that this method can achieve high-precision prediction under various solute-solvent systems and different temperature conditions, and has good generalization ability. It should be noted that although the above embodiments provide specific parameter settings and implementation steps, the present invention is not limited thereto. Any improvement or deformation of the above technical solutions without departing from the idea of the present invention should be regarded as an equivalent alternative of the present invention, and thus fall within the protection scope of the present invention.
Claims
1. A solubility prediction method based on deep learning, characterized in that, Couple the directed message passing neural network DMPNN with the adaptive mixture of experts MoE model to achieve accurate prediction of solubility in various solute-solvent systems and under different temperature conditions. The method includes: (1) Represent the molecular structures of the solute and the solvent as graph structures, where nodes represent atoms and edges represent chemical bonds, and perform feature encoding on them; (2) Use the dual DMPNN model to perform message passing and feature extraction on the molecular graphs of the solute and the solvent respectively, and capture the key information of the molecular structure using graph neural networks; (3) Fuse the obtained molecular feature vectors of the solute and the solvent with environmental information and other molecular descriptors to form a unified input representation; (4) Feed the input representation into the MoE model for solubility prediction. The MoE model includes multiple expert networks and a gating network. The gating network dynamically calculates the weights of each expert network according to the input data and performs weighted fusion on the outputs of the expert networks.
2. The method according to claim 1, wherein The method represents the molecular structures of the solute and the solvent using molecular graph data, and the molecular graph includes node and edge features.
3. The method according to claim 2, wherein The feature representation of the molecular graph data is updated through DMPNN, so as to more accurately capture the chemical properties in the molecular structure.
4. The method according to claim 3, characterized in that Fuse the updated molecular graph features with temperature information and molecular descriptors to form a comprehensive input representation.
5. The method according to claim 4, wherein Feed the comprehensive input representation into the MoE module. This module dynamically generates the weights of each expert network according to the input data through the gating network, and performs weighted fusion on the outputs of each expert network, and then generates the prediction result of solubility.
6. The method according to claim 1, characterized in that, The method is applicable to solubility prediction in various solute-solvent systems and under different temperature conditions, and can achieve better prediction performance than traditional methods on large-scale data sets.
7. The method according to claim 1, characterized in that, The solubility prediction method adopts a cross-validation strategy for training and evaluation to ensure the stability and robustness of the model on different data subsets.
8. A computer-readable storage medium for implementing the method according to claim 1, characterized in that, The storage medium stores a computer program, and when the program is executed on a computer, it can implement the solubility prediction method described in claim 1.
9. A system for implementing the method described in claim 1, the system includes: (1) A data input module for inputting molecular data of the solute and the solvent, temperature information, and molecular descriptors; (2) A feature extraction module for converting the molecular data of the solute and the solvent into a molecular graph with node and edge features; (3) A dual DMPNN module for performing message passing and feature fusion on the molecular graphs of the solute and the solvent respectively; (4) A MoE module, which includes multiple parallel expert networks and a gating network. The gating network dynamically generates the weights of each expert network according to the comprehensive input, and performs weighted fusion on their outputs to obtain the solubility prediction result; (5) An output module for outputting the final solubility prediction value.
10. The system according to claim 9, wherein The molecular graph-level representation of the dual DMPNN module is generated through the following steps: First, pass and update the edge hidden state between neighborhoods through directed edges, then aggregate the incoming edge information for each node and calculate the final node representation through a learnable weight matrix, and finally perform an aggregation operation on the hidden states of all nodes to obtain the molecular graph feature vector.
Citation Information
Patent Citations
A method for predicting the water solubility level of small molecule compounds using deep learning technology
CN109741797B
A method for predicting water solubility of chemical molecules based on deep learning
CN114093435B
A model for predicting the solubility of compound molecules and its application.
CN114334022B
Cited By
ORC working medium thermophysical property prediction method based on graph neural network
CN121506293A
Solute-solvent interaction prediction method based on conditional factor subgraph recognition
CN122177270A
