Molecular property prediction method driven by multi-modal feature fusion reinforcement learning

Abstract

This invention relates to the field of molecular property prediction, specifically disclosing a method for molecular property prediction driven by multimodal feature fusion through reinforcement learning. The method includes loading and preprocessing molecular data; processing the SMILES expressions in the data items, converting them into multimodal feature vectors that can be received by a multilayer perceptron agent; merging multiple feature information through feature fusion and inputting it into the agent participating in meta-training; reconstructing the environment rules and reward function for DiscoRL meta-training; furthermore, using a dynamic equilibrium sampling method for the agent's molecular sampling strategy in the environment rules; dynamically monitoring the validation set performance of the rules by continuously extracting stage-specific meta-network weights during training and embedding them into a fast adaptation loop, recording and saving the best-performing rule weights; this invention significantly outperforms current state-of-the-art methods based on large language models on the MoleculeNet molecular property prediction benchmark, and has significant implications for research in the field of drug discovery.

Description

Technical Field

[0001] This invention belongs to the field of molecular property prediction, specifically involving a molecular property prediction method driven by multimodal feature fusion and reinforcement learning. Background Technology

[0002] Molecular property prediction is fundamental to AI-driven drug discovery. Graph Neural Networks (GNNs) are currently the mainstream technique for handling this type of problem. However, GNN performance heavily relies on large-scale, high-quality labeled data. In drug discovery, scarce and noisy experimental data easily leads to overfitting or poor generalization. Furthermore, pure GNNs struggle to capture macroscopic physical properties, thus encountering a representational bottleneck and resulting in cognitive blind spots in certain tasks. Recently, the introduction of DiscoRL (Discovery Reinforcement Learning) has revolutionized the traditional research paradigm of manually designing reinforcement learning rules. Machines can perform meta-learning through the accumulated experience of multiple agents in large, complex environments, thereby autonomously discovering state-of-the-art reinforcement learning rules. DiscoRL is efficient and scalable, applicable to a wide variety of task environments. However, when a limited, small-scale molecular space is introduced, its meta-training process often falls into memory overfitting. Summary of the Invention

[0003] To address the technical problems mentioned above, this invention provides a molecular property prediction method driven by multimodal feature fusion, which improves the stability and accuracy of model prediction of molecular properties through pre-trained graph neural networks and multimodal feature fusion, K-Fold cross-validation, multi-agent rapid adaptation, and deep ensemble methods.

[0004] To achieve the above technical objectives, the technical solution adopted by this invention is: a molecular property prediction method driven by multimodal feature fusion and reinforcement learning, comprising the following steps:

[0005] S1. Loading and preprocessing of molecular data; pre-training a graph neural network for handling related tasks to extract key topological information from molecules;

[0006] S2. Process the SMILES (Simplified Molecular-Input Line-Entry System) expressions in the data items, and convert them into multimodal feature vectors of Morgan fingerprint, physicochemical property descriptor and MACCS key fingerprint that can be received by the multilayer perceptron agent. Combine multiple feature information through feature fusion and input them into the intelligence participating in meta-training.

[0007] S3. Reconstruct the environment rules and reward function for DiscoRL meta-training; in addition, the dynamic equilibrium sampling method is used for the policy of sampling molecules for the agent in the environment rules.

[0008] S4. Design a new meta-training scheme, which dynamically monitors the validation set performance of rules by continuously extracting meta-network weights at different stages during the training process and embedding them into a fast adaptation loop, and records and saves the rule weights with the best performance.

[0009] The specific operation of S4 includes the following steps:

[0010] S41. Improvements were made to the original rapid adaptation loop process of DiscoRL;

[0011] S42. After the meta-training reaches the set number of steps and reward value, extract RL rules at intervals of steps and perform rapid adaptation, that is, use the rules to train new agents and complete the validation set index test; compare the best index value obtained by the agent trained by the rules at each stage, and save the reinforcement learning rule of the best training session.

[0012] Specifically, the S1 pre-trained graphical neural network includes the following steps:

[0013] S11. Define atomic number, degree, charge, chirality, hybridization type, and aromaticity; encode the atom type, chirality information, and bond type, and concatenate and normalize the codes; use RDKit to read the molecule, traverse the atoms and chemical bonds, and convert it into a PyTorchGeometric data object;

[0014] S12. Use GINClassifier to define a GNN model, construct GINEncoder (custom encoder) to convert the topological graph into a fixed-length vector, and provide the get_embedding function for feature extraction.

[0015] S13. Fine-tune the GNN using a loop, which includes batch loading, forward propagation, loss calculation, and backpropagation. In addition, a balanced weight sampling strategy is used when sampling positive and negative sample molecular data for training.

[0016] Specifically, in S13, the GNN fine-tuning loop uses KL divergence to calculate the overlap between graphs, and the optimizer uses negative log-likelihood loss to calculate the gradient, thereby updating the network weights.

[0017] The training ends when the validation set is run to calculate the metrics for early stopping, and a random weight balancing strategy is used to prevent overfitting.

[0018] Specifically, feature fusion in S2 includes the following steps:

[0019] S21. Perform traditional feature extraction: Using the library functions provided by RDKit, convert the SMILES expressions in the data into feature vectors such as Morgan fingerprints and physicochemical descriptors, and then perform post-processing on these data.

[0020] S22. Perform configuration and dimension calculations to determine the total length of the final output vector. Then, perform multimodal feature pre-fusion. Use the get_embedding function to extract the molecular feature vector obtained by GNN and concatenate it with the traditional feature vector group.

[0021] Specifically, the data processing in S21 includes at least numerical cleaning, security checks, and missing value imputation.

[0022] Specifically, in S4, the obtained reinforcement learning rules are used to quickly train new agents and are used for metric testing to dynamically monitor the validation set performance of the rules.

[0023] Specifically, S41 introduces an early stopping mechanism during the adaptation phase to prevent learners from overfitting. Its workflow includes environment interaction and parameter updates, monitoring validation set performance during training, and performing early stopping and weight saving based on validation set performance evaluation.

[0024] Specifically, the construction of environmental rules in S3 includes state, output action, and round termination condition: the state is set as the observation value Ot received by the agent at each step t: the molecular feature vector fused and normalized by GNN, Morgan and RDKit; the output action is set as the agent needs to make a judgment on the current molecular properties, and in the regression task, continuous physical quantities are mapped to K ordered discrete bucket indices; the termination condition is set as the length of an episode.

[0025] Specifically, the reward function design in S3 is as follows: For the bucketing strategy of regression tasks, a Gaussian kernel reward function is introduced, and a smooth reward gradient is constructed in the discrete action space by adjusting the tolerance σ.

[0026] Specifically, the dynamic balancing sampling method in S3 works as follows: For classification tasks, the distribution of positive and negative samples in the training set is analyzed, the imbalance rate is calculated, and when the ratio reaches or exceeds the threshold, a forced balancing sampling strategy is activated to separate the positive and negative sample pools at the bottom layer of the environment. In each state transition, the next molecular sample is forcibly drawn into the agent at a balanced ratio. If the ratio is lower than the threshold, molecules are sampled according to a random sampling strategy.

[0027] For regression tasks, we use equal-interval binning and equal-probability bin selection to balance the sampling molecules by dividing the target value range in a balanced manner.

[0028] The advantages of this invention compared to existing technologies are as follows: This method combines GNN with autonomous discovery reinforcement learning algorithms, encompassing multimodal feature extraction and fusion, designing environmental rules and reward functions for molecular property prediction tasks, improving meta-training and fast adaptation loops, K-Fold cross-validation and multi-agent fast adaptation, and deep ensemble strategies. Experimental results show that this method significantly outperforms current state-of-the-art methods based on large language models on the MoleculeNet molecular property prediction benchmark, which is of great significance to research in the field of drug discovery. Attached Figure Description

[0029] Figure 1 This is an optional flowchart of the molecular property prediction method driven by multimodal feature fusion and reinforcement learning according to the present invention;

[0030] Figure 2 This is a diagram illustrating the multimodal feature fusion process according to an embodiment of the present invention.

[0031] Figure 3 This is a flowchart illustrating the process of obtaining reinforcement learning rules through a meta-training loop in an embodiment of the present invention.

[0032] Figure 4 This is a diagram illustrating the K-Fold cross-validation, multi-agent training, and deep ensemble process in an embodiment of the present invention.

[0033] Figure 5 This is a diagram illustrating the predictive behavior analysis of an agent according to an embodiment of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0035] Although the steps in this invention are arranged by reference numerals, this is not intended to limit the order of the steps. Unless the order of the steps is explicitly stated or the execution of a step requires other steps as a basis, the relative order of the steps can be adjusted. It is understood that the term "and / or" as used herein refers to and covers any and all possible combinations of one or more of the associated listed items.

[0036] This invention can discover the reinforcement learning rule with the optimal generalization ability for predicting relevant molecular properties, and use the rule to train a new agent to actually complete the relevant prediction task.

[0037] Please see Figures 1-5 As shown, the molecular property prediction method driven by multimodal feature fusion and reinforcement learning includes the following steps:

[0038] S1. Loading and preprocessing of molecular data; pre-training a graph neural network for related tasks to extract key topological information from molecules. The pre-training of the graph neural network includes the following steps:

[0039] S11. Define atomic number, degree (number of connections), charge, chirality, hybridization type, and aromaticity; perform one-hot encoding (encode the atom type, chirality information, and bond type, such as carbon atom C as [1,0,0…], N as [0,1,0…], etc.), and concatenate the codes and normalize; use RDKit to read the molecule, traverse the atoms and chemical bonds, and convert it into a PyTorchGeometric (PyG) data object;

[0040] S12. Use GINClassifier to define a GNN model (GIN, GraphIsomorphismNetwork), construct GINEncoder to convert the topological graph (nodes and edges) into a fixed-length vector, and provide the get_embedding function for feature extraction.

[0041] S13. Fine-tune the GNN in a loop. The operations within the loop include batch loading, forward propagation, loss calculation, and backpropagation. In addition, when sampling positive and negative sample molecular data for training, a balanced weight sampling strategy is adopted to avoid the model from getting trapped in local optima when dealing with imbalanced datasets.

[0042] S2. Process the SMILES expression in the data item, converting it into feature vectors such as Morgan fingerprint, physicochemical property descriptor, and MACCS key fingerprint that can be received by the multilayer perceptron agent (MLP). Through feature fusion, the multiple feature information is combined and input into the intelligence participating in meta-training. The specific multimodal feature fusion includes the following steps:

[0043] S21. Perform traditional feature extraction, that is, use the library functions provided by RDKit to convert the SMILES expression in the data into feature vectors such as Morgan fingerprints and physicochemical descriptors, and perform post-processing on the data, including numerical cleaning, security checks, and missing value imputation (using the median).

[0044] S22. Perform configuration and dimension calculations to determine the total length of the final output vector, and then perform multimodal feature fusion in the early stage. Use the get_embedding function (obtained from step S12) to extract the molecular feature vector obtained by GNN and concatenate it with the traditional feature vector group. This is intended to enable the agent to learn richer multidimensional features and avoid the representation bottleneck of a single feature.

[0045] S3. Reconstruct the environment rules and reward function for DiscoRL meta-training (for example, adopt the core idea of ​​the C51 algorithm to design the environment rules for the regression task and construct the reward function with a Gaussian kernel function), so that the network originally used to process game strategies can correctly learn the feature information related to molecular properties; in addition, the environment rules for the agent to sample molecules use a dynamic equilibrium sampling method to avoid over-sampling of a certain type of molecule and getting trapped in local optima.

[0046] S4. Design a new meta-training scheme. The new scheme introduces an early stopping mechanism to control the training process. During training, the weights of the meta-network are extracted stage by stage. These weights are the reinforcement learning rules learned by the model and are embedded in a fast adaptation loop (using the obtained reinforcement learning rules to quickly train new agents and use them for metric testing) to dynamically monitor the validation set performance of the rules, record and save the best-performing rule weights, thereby solving the memory overfitting problem that is prone to occur in the meta-training stage under the original architecture and improving the generalization ability of the rules. The specific operation includes the following steps:

[0047] S41. Improve the original fast adaptation loop process of DiscoRL by introducing an early stopping mechanism in the adaptation phase to prevent learner overfitting. Its workflow includes environment interaction and parameter update, monitoring validation set performance during training, and early stopping and weight saving based on validation set performance evaluation.

[0048] S42. After the meta-training reaches a certain number of steps and reward value, every certain number of steps, extract RL rules and perform rapid adaptation, that is, use the rules to train new agents and complete the validation set index test; compare the best index value obtained by the agent trained by the rules at each stage, and save the reinforcement learning rule of the best training session.

[0049] Through the above steps, the method of this invention can obtain the reinforcement learning rule with optimal generalization ability for a certain type of molecular property prediction task (such as BBBP blood-brain barrier permeability). Then, this reinforcement learning rule is applied to train a new agent to complete the actual prediction task. The application strategies include:

[0050] Multi-agent training is performed, and a K-Fold cross-validation strategy is added to improve the utilization of the dataset and alleviate the data scarcity problem for some tasks.

[0051] By employing a deep integration strategy, the stability of prediction results is improved and the prediction variance is reduced by averaging the prediction results of all agents. In addition, the improved fast adaptation loop can record several elite agents with the best generalization ability. By integrating their results, the accuracy of molecular property prediction can be further improved.

[0052] In this embodiment, GNN pre-training is performed at the beginning of the project. DiscoMol rapidly pre-trains a GNN network to extract molecular topological information. The steps include defining atomic properties, one-hot encoding, traversing atoms and chemical bonds to convert them into PYG graph objects, balancing weight sampling, and fine-tuning the training loop. Finally, early stopping saves the trained GNN weights and provides key feature extraction functions. Data loading and preprocessing are also performed. First, regarding the dataset source, all experimental data comes from the MoleculeNet dataset compiled by the Stanford PANDE team. This is a publicly available and recognized benchmark dataset for molecular property prediction. These tasks include classification and regression, covering four domains: physiology, biophysics, physical chemistry, and quantum mechanics. The datasets for each task are strictly divided into training, validation, and test sets according to the skeleton. Second, the information input part of the agent under the original architecture needs to be modified to ensure that the MLP (Multi-Layer Perceptron) agent can correctly extract causal representations. The architecture input to the agent and meta-network is "Observation" (O2). t A floating-point tensor (FMT) is a unified mathematical tensor, not a domain-specific semantic modality. Therefore, as long as the observed states are strictly aligned to the underlying standard floating-point tensor format, the agent can correctly receive the information. Based on its underlying implementation being a JAX array and using a unified floating-point format, this invention designs a feature extraction method specifically for molecular property prediction tasks to process agent input.

[0053] Specifically, this invention normalizes and aligns the extracted multimodal features as one-dimensional vectors, and then concatenates them along the feature dimensions. This design aligns with the ingestion mechanism of the underlying MLP network in terms of tensor shape and data type, enabling the agent to extract causal representations in a unified, high-dimensional, and complex chemical feature space.

[0054] Furthermore, by integrating feature extraction and multimodal feature fusion, in addition to using the GNN feature extraction function (obtained from the previous step) to extract the topological structure information of molecules, the library functions provided by RDKit are also used to further convert the SMILES expression into traditional feature vectors such as Morgan fingerprints. Then, the topological structure features extracted by GNN, the Morgan fingerprint, and the macroscopic physical quantities from RDKit are rigorously normalized and aligned, and then concatenated along the feature dimensions. Feature fusion aims to enable the agent to learn richer multidimensional features, avoiding the representational bottleneck of single features. While GNN is good at extracting topological structures, it is prone to skeleton overfitting in small sample scenarios and has an inherent perceptual blind spot when calculating macroscopic physical quantities (such as total lipophilicity). Meanwhile, while traditional Morgan fingerprints can accurately match local substructures, they are limited by high-dimensional sparsity and lack out-of-distribution generalization ability. And while RDKit physicochemical descriptors provide rigorous macroscopic thermodynamic prior features, they lose the spatial information of molecules. Therefore, this invention constructs this Physics-Informed Multi-view State Space, which integrates physical priors.

[0055] Specifically, the meta-training environment design and construction includes environment rules, reward functions, and sampling strategies. DiscoRL's initial environment rules were designed for game-like tasks, defining the agent's behavior in the meta-training loop through action, reward, and iteration rules. In contrast, this invention redesigns a set of general rules and reward functions for molecular property prediction, classification, and regression tasks, constructing them as the agent's meta-training environment. The environment rules primarily facilitate interaction with the agent, including state (the observation value O received by the agent at each step t). t The algorithm consists of several components: molecular feature vectors, actions (the agent needs to determine the properties of the current molecule; for classification tasks, it determines whether the property exists; for regression tasks, it determines which bucket the value belongs to; the bucketing strategy comes from the C51 algorithm in the DeepMind team's ICML paper "A Distributive Perspective on Reinforcement Learning," the core idea of ​​which is to transform the regression task into a classification task, where the agent can output 10 discrete actions representing 10 buckets), and round termination conditions (e.g., the round ends after the agent continuously identifies 32 molecules). The design formula for the regression task reward function is: original reward value... , here Let ∈{0,1,…,K-1} represent the molecular index sampled at step t during training, and y∈{0,1,…,K-1} represent the bucket index, where K is the total number of buckets and σ is the tolerance that controls the reward distribution. The fractions of y and σ are used together as the exponent of e (denoted by exp); a baseline penalty value is introduced. , representing the reward obtained by simply guessing the average bucket, where This represents the mean of the dataset; subtracting the two gives the final difference reward. The signal is amplified by a scaling factor λ, and the result is clipped to the range [-1, 1] using a clip function to stabilize the reinforcement learning training process. This advantage-like formula design ensures that the agent only receives a positive reward when its prediction performance is better than a simple mean guessing strategy; otherwise, it will be penalized negatively. A dynamic balancing sampling strategy is adopted. In classification tasks, DiscoMol automatically determines the sampling strategy based on the data distribution of the training set. For tasks with highly imbalanced data distributions, such as HIV and ClinTox, this invention separates the positive and negative sample pools at the underlying environment level and forces the next molecular sample to be drawn into the agent at a balanced ratio (i.e., 1:1) in each state transition (Step). For datasets with a relatively balanced positive and negative sample distribution, such as BACE, forced balancing sampling is not used to maintain the natural randomness of sampling. In regression tasks, this invention employs equidistant binning and equal-probability bin selection to significantly improve the problem of low sampling rates for extreme and marginal samples by dividing and sampling the target value range in a balanced manner, making the model's learning of the overall data distribution more comprehensive and robust.

[0056] In this embodiment, reinforcement learning rules are dynamically extracted. Since the original DiscoRL architecture was designed for game tasks, its meta-training process tends to pursue efficient training in order to maximize the sum of the agent's expected discounted rewards in the environment. When faced with extremely imbalanced and data-scarce molecular property prediction tasks, such a strategy inevitably leads to memory overfitting.

[0057] Specifically, in a game environment, exhausting states and memorizing the path to victory is often effective; however, in a chemical space, this can cause the metanetwork to abandon the search for general physical and chemical laws and instead look for "shortcuts" to obtain rewards, such as "mean guessing" or simply capturing coincident local features in the training samples.

[0058] In experiments, this phenomenon manifested as rules that converged to higher meta-training reward values ​​performing poorly when training new agents. Therefore, this invention introduces an early stopping mechanism for meta-training to preserve the set of rules with the best generalization ability during training for subsequent prediction. Specifically, after reaching a certain number of steps and reward values, RL rules are extracted at regular intervals, used to train new agents, and their metrics are tested. Finally, the best metric value obtained by the agent trained with rules at each stage is compared, and the reinforcement learning rule from the best training session is saved.

[0059] Furthermore, K-Fold cross-validation, rapid multi-agent adaptation, and deep ensemble are employed. This step formally applies the discovered reinforcement learning rules to train agents, enabling them to predict molecular properties for specific tasks. During the adaptation phase, for tasks with scarce data, this invention uses K-Fold cross-validation to dynamically partition the dataset among different agents. This strategy fully utilizes the limited dataset while also incorporating early stopping to improve generalization. Regarding the C51 algorithm, since it predicts the probability of falling into each bucket, it is naturally suited to probability-weighted averaging and deep ensemble methods. DiscoMol trains multiple randomly initialized agents, and during the testing phase, the prediction results of these agents are averaged. This invention uses full ensemble and elite ensemble to improve the stability and accuracy of predictions. Full ensemble averages the prediction results of multiple agents involved in training, while elite ensemble averages the prediction results of the agent with the best generalization ability during training.

[0060] In this embodiment, the first stage is the dynamic equilibrium sampling stage, which is used to dynamically generate training samples according to task type and label distribution during model training or rapid adaptation; the second stage is the dynamic K-Fold multi-agent rapid adaptation and ensemble evaluation stage, which is used to improve the model's generalization ability and prediction stability in small sample molecular property prediction tasks.

[0061] In one possible implementation, the molecular dataset D, the corresponding label Y, and the current task type T are first obtained. Task type T includes classification and regression tasks. For classification tasks, the label Y typically represents the molecule's category attribute, such as active or inactive molecules; for regression tasks, the label Y typically represents a continuous molecular property value, such as hydration free energy, solubility, affinity, or other physicochemical properties.

[0062] During the dynamic balancing sampling phase, the system first initializes the label distribution based on the task type T. When the task type T is a classification task, it counts the number of positive class samples Npos and the number of negative class samples Nneg in the dataset, and calculates the class imbalance ratio γ based on the ratio of the majority class samples to the minority class samples. If the class imbalance ratio γ is greater than the preset imbalance trigger threshold τ, it is determined that there is a significant class imbalance in the current dataset, and the forced balancing sampling mechanism is activated; if the class imbalance ratio γ is not greater than the imbalance trigger threshold τ, it is considered that the current positive and negative sample distribution is relatively balanced, the forced balancing sampling mechanism is not activated, and a natural random sampling method is used instead.

[0063] The imbalance trigger threshold τ can be set to 3.5 by default. This means that the system will only initiate forced balancing sampling when the number of majority class samples reaches more than 3.5 times the number of minority class samples. This threshold setting avoids frequent intervention in the original data distribution when there are slight fluctuations in the sample distribution, while increasing the probability of minority class samples appearing during training when there is a severe class imbalance.

[0064] When the forced balanced sampling mechanism is enabled for classification tasks, the system divides the sample indices in the molecular dataset D into a positive class index pool (Ppos) and a negative class index pool (Pneg) according to the label category. During each sampling, the system first randomly determines the target class for this sampling with a probability of 0.5, i.e., selecting either the positive or negative class with equal probability. Then, it uniformly and randomly selects a sample index from the corresponding positive class index pool (Ppos) or negative class index pool (Pneg) as the molecular sample used in this training or adaptation step. This achieves an approximately 1:1 dynamic balance between positive and negative classes at the sampling level, thereby reducing the dominant role of majority class samples in the model update direction.

[0065] When the forced balanced sampling mechanism is not enabled for the classification task, the system no longer distinguishes between the positive and negative class index pools, but instead directly and uniformly randomly draws samples from the entire index range of the molecular dataset D. This approach preserves the original data distribution and is suitable for classification tasks with a relatively reasonable class ratio.

[0066] For regression tasks, since the labels Y are continuous values, it is not possible to directly divide the sample pool according to positive and negative categories. Therefore, the system defaults to using a balanced sampling mechanism and divides the continuous label space into K equal-width intervals B1 to BK. Subsequently, the system counts the number of molecular samples Ck contained in each interval and establishes a mapping relationship Mk between interval numbers and sample index sets for non-empty intervals.

[0067] In regression tasks, molecular property values ​​typically exhibit a distribution pattern of dense samples in the middle region and sparse samples in the extreme value regions at both ends. If sampling is performed directly according to the original data distribution, the model is prone to bias towards the high-frequency property intervals in the middle, while failing to learn the sparse property intervals at the tail. To address this, the system calculates the smoothed sampling probability pk based on the number of samples Ck in each interval. Specifically, the square root of the number of samples Ck in each interval is taken, and then the square root results of all intervals are normalized to obtain the corresponding interval sampling probability pk.

[0068] By applying square root smoothing to the sample size, the difference in sampling probabilities between high-frequency and low-frequency intervals can be compressed, allowing the high-frequency interval to maintain a certain advantage while significantly increasing the sampling opportunity for sparse tail intervals. During each regression task sampling, the system first randomly selects a target interval (target_bin) from multiple intervals based on smoothed sampling probabilities p1 to pK, and then uniformly and randomly extracts a molecular sample index from the corresponding sample index set Mtarget_bin. Thus, the model can more fully engage with molecular samples across different property ranges during training, improving its coverage of the continuous property space.

[0069] After completing the dynamic balancing sampling logic, the system enters the dynamic K-Fold multi-agent rapid adaptation and ensemble evaluation phase. This phase first constructs a support set Dsupp, which is obtained by merging the original training set Dtrain and validation set Dvalid. The test set Dtest is retained independently for the final evaluation of model performance. The system also acquires the parameters Φ of the discovered reinforcement learning rules, the number of agents K, the number of rapid adaptation steps S, the evaluation interval I, the task type T, and the number of elite agents E.

[0070] In the dynamic K-Fold fast adaptation process, the system constructs different data partitions for each agent. For the k-th agent, the system dynamically divides the support set Dsupp into a training subset Dtrain(k) and a validation subset Dvalid(k) based on a preset ratio. In one implementation, the partition ratio can be set to 85:15, and different random seeds can be set according to the agent number k, so that different agents obtain different combinations of data subsets. In this way, although each agent comes from the same support set, their training samples and validation samples are different, thus forming diversity in prediction behavior.

[0071] Subsequently, the system initializes the state and model weights θ(k) of the k-th agent, and records the current best validation score (best_score) and the corresponding best weight (best_θ). The k-th agent performs S-step fast adaptation on its corresponding training subset Dtrain(k). In each adaptation step, the agent performs a FastAdaptation operation according to the reinforcement learning rule parameters Φ, thereby updating the model weights θ(k). This update process is used to enable the agent to quickly adapt to the distribution characteristics of the current data subset within a finite number of steps.

[0072] During rapid adaptation, the system periodically validates the agent according to a preset evaluation interval I. When the current step number is divisible by the evaluation interval I, or when the current step number reaches the final adaptation step S, the system temporarily saves the current adaptation environment state and switches to the validation or test evaluation environment. Subsequently, the system uses the current agent weight θ(k) to calculate the validation metric val_score on the corresponding validation subset Dvalid(k).

[0073] The system employs different optimal score determination rules for different task types. When task type T is a regression task, the evaluation metrics are typically error-related indicators such as RMSE and MAE; in this case, a smaller indicator value indicates better performance. When task type T is a classification task, the evaluation metrics are typically performance-related indicators such as ROC-AUC, accuracy, or F1 score; in this case, a larger indicator value indicates better performance. The system determines whether val_score is better than the historical best score best_score based on the current task type. If the current validation result is better, the current score is updated to best_score, and the current model weights θ(k) are saved as the optimal weights best_θ.

[0074] After completing the S-step fast adaptation, the system restores the model state of the k-th agent to the best weight best_θ corresponding to its historical best performance, instead of directly using the weights obtained in the last adaptation step. This rollback mechanism avoids overfitting or performance degradation that may occur in subsequent adaptation steps, ensuring that each agent participates in the final test with its own best-performing state.

[0075] Subsequently, the system applies the k-th agent, now restored to its optimal state, to the test set Dtest, obtaining the agent's prediction result test_preds(k) on the test set, and storing this prediction result in the prediction result set Pall. Simultaneously, the system stores the agent's corresponding best validation score best_score in the validation score set Vall. The system sequentially performs the above dynamic partitioning, rapid adaptation, periodic validation, optimal state saving, and test prediction steps for the K agents until it obtains the test prediction results and validation performance records for all K agents.

[0076] During the ensemble reasoning phase, the system can generate the final test results using either full ensemble or elite ensemble methods.

[0077] In the full ensemble approach, the system averages the prediction results of K agents on the test set Dtest. For classification tasks, the prediction result can be the predicted probability of each molecule belonging to the target category; for regression tasks, the prediction result can be the predicted value of the continuous properties of each molecule. The system averages the prediction results of all agents sample by sample to obtain the final ensemble prediction result, and calculates the final evaluation index on the test set based on this final ensemble prediction result.

[0078] In the elite ensemble approach, the system first ranks the K agents based on their validation score set Vall, and selects the top E agents as elite agents. For classification tasks, agents with higher validation metrics are prioritized; for regression tasks, agents with lower validation errors are prioritized. Subsequently, the system averages the prediction results of the E elite agents on the test set to obtain the elite ensemble prediction result, and further calculates the final test metric. Compared to full ensemble, elite ensemble can exclude some agents with poor validation performance, reducing the interference of low-quality models on the final prediction result.

[0079] Through the aforementioned data processing logic, this invention can dynamically adjust training samples even when sample categories are imbalanced or the distribution of continuous properties is uneven. Simultaneously, it utilizes dynamic K-Fold partitioning to generate multiple agents with differentiated data perspectives, and improves the generalization performance of individual agents through rapid adaptation and optimal state rollback. Finally, the system integrates the prediction results of multiple agents through either full or elite ensemble methods, thereby improving the stability, robustness, and final evaluation performance in molecular property prediction tasks.

[0080] Figure 2 The process of multimodal feature fusion is demonstrated. First, the dataset input (SMILES expression, Simplified Molecular InputLineEntrySystem) is transformed into relevant feature vectors, including feature vectors extracted by GNN (512-dimensional, representing molecular topological structure information), Morgan fingerprints (high-dimensional, sparse binary vectors, representing the local chemical environment centered on each atom), and RDKit physicochemical descriptors (low-dimensional, dense, continuous numerical vectors, representing the overall physicochemical properties of the molecule). Then, the three sets of feature vectors are concatenated for early multimodal fusion, and finally input into an agent (MLP) with a network architecture of multilayer perceptron for subsequent training.

[0081] Figure 3The diagram illustrates the process of the meta-training loop and the extraction of reinforcement learning rules. The core step of the meta-training loop is the interaction between the agent (MLPAgent) and the environment (Env). The agent samples molecules according to the sampling strategy in the environment (such as the Sampling Strategy, a balanced sampling strategy), obtains feature information through the rendering function (which returns a one-dimensional array of feature vectors corresponding to the molecules), and then outputs the prediction result. The reward value is calculated through the reward function (such as the Gaussian kernel function as the reward). Then, the output prediction value, reward value, round termination signal, and other information are used as a trajectory and input to the meta-network. The meta-network obtains the update target and updates its own and the agent's network weights accordingly. During this process, the meta-network weights are intermittently extracted to train new agents for validation set metric testing. Finally, the set of network weights with the best metric test is selected, which is the discovered reinforcement learning rule.

[0082] Figure 4 This paper demonstrates the processes of K-Fold cross-validation, multi-agent rapid adaptation, and deep ensemble. As shown in the figure, the training and validation sets are fused and then divided. With K agents participating in rapid adaptation, the dataset is divided into K groups, each with distinct training and validation sets. Agents are then assigned to these groups for rapid adaptation (reinforcement learning rules are applied to update network weights; training is performed on the training set, and early stopping is performed on the validation set). A number of elite agents are selected (ranked from Top1, Top2, up to TopM based on validation set performance, where M < K). Test set predictions are then used for evaluation. The prediction results are then used for deep ensemble. The diagram illustrates the basic principle of deep ensemble: if Voter1 predicts a molecular property value with a probability of 0.9 falling in bucket 2 and 0.1 falling in bucket 3, the final output is 0.9 x 2 + 0.1 x 3 = 2.1. This process is repeated for all Voters, and the results are averaged to obtain the final result. Full ensemble averages the predictions of all K agents, while elite ensemble averages the predictions of the top M agents.

[0083] Table 1. Characteristics and performance of various variant models: ablation experimental results

[0084]

[0085] Table 2 Performance Comparison Results of Different Sampling Strategies

[0086]

[0087] Table 3. Results of ablation experiments using the early cessation strategy.

[0088]

[0089] Table 4. Performance comparison results of ensemble strategy and K-fold cross-validation

[0090]

[0091] It should be noted that in Tables 1 to 4, √ indicates that the feature / strategy is enabled, and × indicates that it is not enabled; ROC-AUC↑ indicates that the higher the metric, the better the model performance.

[0092] The effectiveness of each strategy in the method of this invention is demonstrated through the analysis of Tables 1 to 4. The first test metric is ROC-AUC, or Area Under the Receiver Operating Characteristic Curve, one of the core evaluation metrics for molecular property prediction tasks. A higher score is better, expressed as a percentage. The tasks shown include BBBP (blood-brain barrier permeability), BACE (β-secretase 1 inhibitor prediction), HIV (HIV replication inhibition activity prediction), and ClinTox (clinical trial toxicity prediction). The first table shows the ablation experiment results of multimodal fusion. For example, the first row indicates that the RDKit physicochemical descriptor was removed, and only the features extracted by GNN were fused with Morgan fingerprint features. The performance of this metric decreased compared to full fusion (last row) on some tasks, demonstrating the necessity of fusing RDKit physicochemical descriptor features. "Pure image features only" indicates the use of only GNN-extracted features, demonstrating the importance of fusing traditional features, while "traditional GNN" indicates the use of only GNN features. The first table uses N to predict molecular properties without using DiscoMol, serving as a baseline for comparison to demonstrate the effectiveness of the method. The second table shows whether the forced balanced sampling strategy is used for each task, and the corresponding scores. The third table shows the ablation experiment results of the early stopping strategy in the training loop. Removing the early stopping strategy in both the meta-training and adaptation phases results in a decrease in scores. The fourth table shows the ablation experiment results of K-Fold cross-validation and deep ensemble strategies, displaying the experimental results under simple averaging, full-ensemble, and elite-ensemble multi-agent strategies. Each item shows the impact of using and not using the K-Fold strategy, and the bottom row represents the final prediction score.

[0093] Figure 5The distribution of feature saliency that the agent focuses on in the BBBP task is shown. Further investigation into the feature dependencies between positive and negative samples is presented, as shown in the figure (bottom right). Positive samples (red area) exhibit sharp, high-intensity gradient peaks, while negative samples (blue area) show a smoother and more dispersed dominance distribution. By comparing the common and differential features of positive samples (BBBP=1) and negative samples (BBBP=0), this invention reveals the asymmetric reasoning logic exhibited by the agent in the classification task. The two-color aggregated histogram at the bottom shows that at the gradient peaks, the saliency of the common features of positive samples (red area) is significantly higher than that of negative samples (blue area). This indicates that DiscoMol is highly sensitive and, when learning the blood-brain barrier penetration rules, focuses on the positive pharmacophores (e.g., specific lipophilic groups) that determine permeability. However, when identifying impenetrable negative sample molecules, the model's attention is relatively dispersed due to the diversity of negative molecules. The left column of the figure shows the saliency plots for six correctly predicted positive molecules, while the right column shows the saliency plots for a mixture of three negative molecules and three positive molecules. Notably, the key peak driving the agent to make a positive prediction is located in dimension 1776, with a significance value exceeding 0.06. The molecule at index 1 (true label = 0) has a value of 0.062 in this dimension, close to the peak, and is therefore misclassified as positive. Conversely, the molecule at index 15 is a true positive molecule, but because its significance value in this key dimension is only 0.044, it is misclassified as negative. The saliency analysis of the mispredicted negative samples reveals the potential limitations of meta-networks in handling extremely complex feature mappings.

[0094] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A molecular property prediction method driven by multimodal feature fusion and reinforcement learning, characterized in that, Includes the following steps: S1. Loading and preprocessing of molecular data; pre-training a graph neural network for handling related tasks to extract key topological information from molecules; S2. Process the SMILES expression in the data item and convert it into a multimodal feature vector of Morgan fingerprint, physical and chemical property descriptor and MACCS key fingerprint that can be received by the multilayer perceptron agent. Then, merge the multiple feature information through feature fusion and input it into the intelligence participating in meta-training. S3. Reconstruct the environment rules and reward function for DiscoRL meta-training; in addition, the dynamic equilibrium sampling method is used for the policy of sampling molecules for the agent in the environment rules. S4. Design a new meta-training scheme, which dynamically monitors the validation set performance of rules by continuously extracting meta-network weights at different stages during the training process and embedding them into a fast adaptation loop, and records and saves the rule weights with the best performance. The specific operation of S4 includes the following steps: S41. Improvements were made to the original rapid adaptation loop process of DiscoRL; S42. After the meta-training reaches the set number of steps and reward value, extract RL rules at intervals of steps and perform rapid adaptation, that is, use the rules to train new agents and complete the validation set index test; compare the best index value obtained by the agent trained by the rules at each stage, and save the reinforcement learning rule of the best training session.

2. The molecular property prediction method driven by multimodal feature fusion and reinforcement learning according to claim 1, characterized in that, The S1 pre-trained graphical neural network includes the following steps: S11. Define atomic number, degree, charge, chirality, hybridization type, and aromaticity; encode the atom type, chirality information, and bond type, and concatenate and normalize the codes; use RDKit to read the molecule, traverse the atoms and chemical bonds, and convert it into a PyTorchGeometric data object; S12. Use GINClassifier to define a GNN model, construct GINEncoder to convert the topological graph into a fixed-length vector, and provide the get_embedding function for feature extraction. S13. Fine-tune the GNN using a loop, which includes batch loading, forward propagation, loss calculation, and backpropagation. In addition, a balanced weight sampling strategy is used when sampling positive and negative sample molecular data for training.

3. The molecular property prediction method driven by multimodal feature fusion and reinforcement learning according to claim 2, characterized in that, In S13, the GNN fine-tuning loop uses KL divergence to calculate the overlap between graphs, and the optimizer uses negative log-likelihood loss to calculate the gradient, thereby updating the network weights. The training ends when the validation set is run to calculate the metrics for early stopping, and a random weight balancing strategy is used to prevent overfitting.

4. The molecular property prediction method driven by multimodal feature fusion and reinforcement learning according to claim 2, characterized in that, Feature fusion in S2 includes the following steps: S21. Perform traditional feature extraction: Using the library functions provided by RDKit, convert the SMILES expressions in the data into Morgan fingerprints and physicochemical descriptor feature vectors, and then perform post-processing on these data. S22. Perform configuration and dimension calculations to determine the total length of the final output vector. Then, perform multimodal feature pre-fusion. Use the get_embedding function to extract the molecular feature vector obtained by GNN and concatenate it with the traditional feature vector group.

5. The molecular property prediction method driven by multimodal feature fusion and reinforcement learning according to claim 2, characterized in that, Data processing in S21 includes at least numerical cleaning, security checks, and missing value imputation.

6. The molecular property prediction method driven by multimodal feature fusion and reinforcement learning according to claim 2, characterized in that, The obtained reinforcement learning rules are used in S4 to quickly train new agents and are used for metric testing to dynamically monitor the validation set performance of the rules.

7. The molecular property prediction method driven by multimodal feature fusion and reinforcement learning according to claim 6, characterized in that, S41 specifically introduces an early stopping mechanism during the adaptation phase to prevent learner overfitting. Its workflow includes environment interaction and parameter updates, monitoring validation set performance during training, and performing early stopping and weight saving based on validation set performance evaluation.

8. The molecular property prediction method driven by multimodal feature fusion and reinforcement learning according to claim 2, characterized in that, The construction of environmental rules in S3 includes state, output action, and round termination condition: the state is set as the observation value O received by the agent at each step t. t The molecular feature vector is obtained by fusing and normalizing GNN, Morgan fingerprint and RDKit; the output action is set as the agent needs to make a judgment on the current molecular property. In the regression task, the continuous physical quantity is mapped to K ordered discrete bucket indices; the termination condition is set as a specified episode length.

9. The molecular property prediction method driven by multimodal feature fusion and reinforcement learning according to claim 7, characterized in that, The reward function design in S3: For the bucketing strategy of regression tasks, a Gaussian kernel reward function is introduced. By adjusting the tolerance σ, a smooth reward gradient is constructed in the discrete action space.

10. The molecular property prediction method driven by multimodal feature fusion and reinforcement learning according to claim 2, characterized in that, The dynamic balancing sampling method in S3: For classification tasks, analyze the distribution of positive and negative samples in the training set, calculate the imbalance rate, and when the ratio reaches or exceeds the threshold, activate the forced balancing sampling strategy to separate the positive and negative sample pools at the bottom layer of the environment. In each state transition, force the next molecular sample to be drawn into the agent at a balanced ratio. If the ratio is lower than the threshold, sample molecules according to the random sampling strategy. For regression tasks, we use equal-interval binning and equal-probability bin selection to balance the sampling molecules by dividing the target value range in a balanced manner.

Images