A molecular representation learning system enhanced by task description

Abstract

A molecular representation learning system enhanced by task description. The present invention belongs to the field of molecular representation learning in artificial intelligence in the computer field, and specifically relates to a molecular representation learning system enhanced by task description. The present invention aims to solve the problem in the prior art that general molecular representations are difficult to capture task-specific features and have insufficient generalization performance in scenarios where labeled data is scarce, resulting in low accuracy and poor interpretability of molecular representation learning. The system includes: a multi-regression task dataset construction module based on task description, a dual-tower model construction module, a dual-tower model training module, and a prediction module; the multi-regression task dataset construction module based on task description is used to construct a multi-regression task dataset based on task description; the dual-tower model construction module is used to construct a dual-tower model; the dual-tower model training module is used to obtain a trained dual-tower model; the prediction module is used to input the molecules to be tested into the trained dual-tower model, and the trained dual-tower model outputs task-related molecular representations.

Description

Technical Field

[0001] The present invention belongs to the field of molecular representation learning in artificial intelligence in the computer field, and specifically relates to a molecular representation learning system with enhanced task description. Background Art

[0002] With the rapid development of artificial intelligence (AI), deep learning has been widely applied in drug discovery and chemistry. Molecular representation learning has become a key research direction, particularly in tasks such as compound screening and drug interaction prediction. The core of molecular representation lies in converting molecular structures into mathematical vector representations, enabling them to be used by machine learning models for predictions in downstream tasks.

[0003] Traditional molecular characterization methods include molecular descriptors and molecular fingerprints constructed based on expert knowledge, such as Morgan fingerprints (Morgan HL (1965) The generation of a unique machine description for chemical structures—a technique developed at chemical abstracts service. J Chem Doc 5:107–113. https: / / doi.org / 10.1021 / c160017a018) and ECFP4 (Rogers D, Hahn M (2010) Extended-connectivity fingerprints. J Chem Inf Model 50:742–754. https: / / doi.org / 10.1021 / ci100050t). Although these methods have strong interpretability, they have obvious limitations in capturing complex molecular features. With the rise of deep learning, many studies have begun to use neural networks to learn molecular representations from labeled data (Gilmer J, Schoenholz SS, Riley PF, et al. Neural message passing for quantum chemistry [C] / / Proceedings of the 34th International Conference on Machine Learning. PMLR, 2017: 1263-1272.), (Yang K, Swanson K, Jin W, et al. Analyzing learned molecular representations for property prediction [J]. Journal of chemical information and modeling, 2019, 59 (8): 3370-3388.). These methods have shown significant advantages in specific tasks, but due to the scarcity of high-quality labeled data, the performance and generalization ability of the models are often difficult to break through the bottleneck.

[0004] In recent years, the success of self-supervised learning models, especially pre-trained language models in natural language processing, has triggered a research boom in applying pre-trained models to molecular representation learning (Devlin J, Chang MW, Lee K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding [C] / / Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019: 4171-4186.). Molecules are represented by sequences or graph structures and pre-trained using self-supervised tasks similar to language models. This pre-training paradigm significantly improves the representation ability and generalization of the model (Chithrananda S, Grand G, Ramsundar B. ChemBERTa: Large-scale self-supervised pretraining for molecular property prediction [J]. arXiv preprint arXiv: 2010.09885, 2020.), (Rong Y, Bian Y, Xu T, et al. Self-supervised graph transformer on large-scale molecular data [C] / / Advances in Neural Information Processing Systems. 2020, 33: 12559-12571.).At the same time, some methods introduce more external information (such as knowledge graphs (Fang X, Liu Y, Chen L, et al. Knowledge graph contrastive learning for molecular property prediction [C] / / Proceedings of the 2022 SIAM International Conference on Data Mining (SDM). SIAM, 2022: 244-252.) or text data (Zeng D, Zhang J, Li R, et al. KV-PLM: A knowledge-enhanced pre-trained language model for molecular property prediction [J]. arXiv preprint arXiv:2205.00464,2022.) to enhance molecular representation. Existing molecular representation learning methods have achieved remarkable results in generating universal molecular representations. Although universal molecular representations can be used for a variety of tasks, they may not always be the best fit for a specific task. This is because different molecular property prediction tasks often require attention to different molecular features. For example, toxicity prediction may require a focus on known toxic substructures, while binding affinity prediction requires special attention to structural features that interact with specific protein targets. Universal representations struggle to capture the detailed features specific to these tasks.

[0005] In this context, molecular representation learning methods enhanced by task descriptions demonstrate unique advantages and significant value. First, by introducing task descriptions to assist molecular representation learning, the model can better distinguish between key features and secondary features in a specific task. This task-aware capability is particularly important for scenarios where labeled data is scarce and can significantly improve the generalization performance of the model. For example, in the task of drug-target interaction prediction, the model can automatically adjust its focus on regions in the molecular structure related to the binding site based on the descriptive information of the target. Second, task-related molecular representations have obvious advantages in computational efficiency and interpretability. Since molecular representations are optimized for specific tasks, models can often achieve better performance using the same parameters and computing resources. At the same time, the embedded representations generated by this method are more interpretable and can establish a clear association between specific structural features of the molecule and the task results. This is of great significance for understanding the reasons for the prediction results and guiding molecular optimization design.

[0006] To address the above issues, the present invention proposes a new method that combines the natural language understanding capabilities of language models and uses text prompts to guide the generation of molecular representations, enabling the model to adjust molecular features according to task requirements, thereby generating task-related molecular representations and improving the performance of downstream tasks. Summary of the Invention

[0007] The purpose of this invention is to solve the problems in the prior art that general molecular representations are difficult to capture task-specific features and have insufficient generalization performance in scenarios with scarce labeled data, resulting in low accuracy and poor interpretability of molecular representation learning, and to propose a molecular representation learning system with enhanced task description.

[0008] A molecular representation learning system with enhanced task description includes:

[0009] Multi-regression task dataset construction module based on task description, dual-tower model construction module, dual-tower model training module, and prediction module;

[0010] The multi-regression task dataset construction module based on task description is used to construct the multi-regression task dataset MT-MTR based on task description. The specific process is as follows:

[0011] First, obtain molecules from the DrugBank and ChEBI databases, and use RDKit to normalize the molecules in the DrugBank and ChEBI databases to obtain standardized SMILES format molecules;

[0012] Deduplication is performed on the molecules in the normalized SMILES format to obtain the normalized SMILES format molecules after deduplication;

[0013] 2. Get a list of molecular descriptors;

[0014] The molecular descriptor list contains the names of each molecular descriptor and the functions for calculating each molecular descriptor;

[0015] 3. For each molecule obtained in step 1, randomly select K descriptor names and calculation functions corresponding to the descriptor names from the molecular descriptor list obtained in step 2;

[0016] Calculate the properties of each molecule using K randomly selected calculation functions;

[0017] The K attributes of each molecule serve as K regression labels for the molecular descriptor;

[0018] Input K randomly selected descriptor names into the large language model, and the large language model outputs a task description;

[0019] Fourth, construct triplet data (molecule, task description, regression label) based on the first obtained molecule, the third obtained task description and the regression label, and construct the multi-regression task dataset MT-MTR based on the triplet data;

[0020] The twin tower model building module is used to build a twin tower model;

[0021] The dual-tower model training module is used to obtain the trained dual-tower model;

[0022] The prediction module is used to input the molecules to be tested into the trained dual-tower model, and the trained dual-tower model outputs task-related molecular representations.

[0023] The beneficial effects of the present invention are:

[0024] The present invention provides a molecular representation learning method with enhanced task description based on a language pre-training model and a molecular pre-training model.

[0025] This method uses textual cues to adjust molecular representations, making them more suitable for specific downstream tasks, thereby improving prediction accuracy and training efficiency. This technology can be widely used in drug discovery, molecular property prediction and other fields.

[0026] This paper proposes a method for learning molecular representations enhanced with task descriptions. By combining task descriptions with molecular representations, this method enhances the task specificity of molecular representations. This method not only incorporates the knowledge contained in the text but also leverages the reasoning capabilities of language models, providing new research ideas for molecular representation learning.

[0027] 1. To address the difficulty of capturing task-specific features in existing general molecular representations, this paper introduces task descriptions to guide the generation of molecular representations. Experimental results demonstrate that this approach achieves significant improvements on multiple regression tasks, achieving state-of-the-art performance on the ESOL, FreeSolv, and Lipophilicity datasets.

[0028] 2. To address the problem of insufficient generalization performance in scenarios with scarce labeled data, this paper constructs a multi-regression task dataset (MT-MTR) based on task descriptions and designs corresponding pre-training tasks. Experimental verification demonstrates that this method not only focuses on relevant features explicitly mentioned in the task description, but also identifies unmentioned but relevant features, while reducing attention to irrelevant features, demonstrating excellent generalization capabilities.

[0029] 3. To address the issue of poor interpretability, this paper verifies that the model indeed pays attention to key task-related information by analyzing the attention matrix. For example, in the solubility prediction task, the model focuses on tokens closely related to solubility, such as "groups" and "ethers," as well as polar functional groups in molecules.

[0030] 4. This method has good transferability, achieving similar performance improvements on different molecular pre-trained models (such as ChemBERTa-2 and CHEM-BERT). This demonstrates that this method can inherit the advantages of different underlying models and has broad application prospects. This method can be applied to molecular property prediction tasks in fields such as drug discovery and materials design. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of the overall system solution of the present invention;

[0032] Figure 2 This is the system model diagram of the present invention. DETAILED DESCRIPTION

[0033] Specific embodiment 1: This embodiment provides a molecular representation learning system with enhanced task description, including:

[0034] Multi-regression task dataset construction module based on task description, dual-tower model construction module, dual-tower model training module, and prediction module;

[0035] The multi-regression task dataset construction module based on task description is used to construct the multi-regression task dataset MT-MTR based on task description. The specific process is as follows:

[0036] 1. Obtain molecules from the DrugBank (Knox C, Wilson M, Klinger CM, et al. DrugBank 6.0: the DrugBank knowledgebase for 2024[J]. Nucleic acids research, 2024, 52(D1): D1265-D1275.) and ChEBI (Hastings J, Owen G, Dekker A, et al. ChEBI in 2016: Improved services and an expanding collection of metabolites[J]. Nucleic acids research, 2016, 44(D1): D1214-D1219.) databases, and use RDKit to normalize the molecules in the DrugBank and ChEBI databases to obtain standardized SMILES format molecules;

[0037] Deduplication is performed on the molecules in the normalized SMILES format to obtain the normalized SMILES format molecules after deduplication;

[0038] 2. Get a list of molecular descriptors;

[0039] The molecular descriptor list contains the names of each molecular descriptor and the functions for calculating each molecular descriptor;

[0040] 3. For each molecule obtained in step 1, randomly select K (5-10) descriptor names and calculation functions corresponding to the descriptor names from the molecular descriptor list obtained in step 2;

[0041] Calculate the properties of each molecule using K randomly selected calculation functions;

[0042] The K attributes of each molecule serve as K regression labels for the molecular descriptor;

[0043] Input K randomly selected descriptor names into the large language model, and the large language model outputs a task description;

[0044] The regression label consists of two 209-dimensional vectors. The first vector stores the value of the regression label, and the second vector consists of 0 and 1, where 0 indicates missing and 1 indicates the presence of the regression label. There are K 1s in the second vector.

[0045] Fourth, construct triplet data (molecule, task description, regression label) based on the first obtained molecule, the third obtained task description and the regression label, and construct the multi-regression task dataset MT-MTR based on the triplet data;

[0046] The twin tower model building module is used to build a twin tower model;

[0047] The dual-tower model training module is used to obtain the trained dual-tower model;

[0048] The prediction module is used to input the molecules to be tested into the trained dual-tower model, and the trained dual-tower model outputs task-related molecular representations.

[0049] Specific embodiment 2: This embodiment differs from specific embodiment 1 in that: in the second embodiment, a molecular descriptor list is obtained; the specific process is:

[0050] Use Descriptors in RDKit to get a list of all molecular descriptors supported by RDKit

[0051] Descriptors.descList;

[0052] The molecular descriptor list contains 209 molecular descriptor names and functions for calculating each molecular descriptor; including but not limited to the following features: maximum absolute electronic state index (MaxAbsEStateIndex), maximum electronic state index (MaxEStateIndex), minimum absolute electronic state index (MinAbsEStateIndex), minimum electronic state index (MinEStateIndex), quantitative drug similarity score (QED, Quantitative Estimate of Drug-likeness), molecular weight (MolWt, Molecular Weight);

[0053] 1) Molecular data were obtained from DrugBank and ChEBI databases, and 55,759 valid SMILES were obtained after deduplication and normalization using RDKit.

[0054] 2) Calculate 209 molecular properties of each molecule using RDKit;

[0055] 3) Randomly sample 5-10 properties for each molecule, generate task descriptions using the sampled property names through GPT-3.5, and use the corresponding property values ​​as regression labels;

[0056] 4) Finally, construct a (molecule, task description, regression label) triplet dataset.

[0057] Other steps and parameters are the same as those in the first embodiment.

[0058] Specific embodiment three: This embodiment differs from specific embodiment one or two in that: the twin-tower model construction module is used to construct a twin-tower model; the specific process is:

[0059] The dual-tower model includes a text encoder (T-Encoder) and a molecular encoder (M-Encoder);

[0060] The text encoder (T-Encoder) is the language pre-training model PubMedBERT;

[0061] The molecular encoder (M-Encoder) is a molecular pre-training model CHEM-BERT;

[0062] The language pre-training model PubMedBERT is divided into a unimodal text encoder and a multimodal text encoder;

[0063] The unimodal text encoder is the 1st to 9th layers in the language pre-training model PubMedBERT;

[0064] The multimodal text encoder is the 10th to 12th layers in the language pre-training model PubMedBERT;

[0065] Each self-attention layer in the multimodal text encoder is replaced with a cross-attention layer to achieve the fusion of task description embedding and molecular representation.

[0066] The unimodal encoder is used to understand the semantic information of the task description, and the multimodal encoder is used to adjust the molecular representation.

[0067] Other steps and parameters are the same as those in the first or second embodiment.

[0068] Specific embodiment 4: This embodiment differs from specific embodiments 1 to 3 in that:

[0069] The model training module is used to obtain the trained dual-tower model; the specific process is:

[0070] 1) The molecules in the triplet data are input into the molecular encoder, and the molecular encoder outputs the universal molecular representation; the universal molecular representation is mapped through the linear layer to obtain a vector (K m ,V m );

[0071] 2) The task description in the triple data is input into the unimodal text encoder, and the unimodal text encoder outputs the representation of the task description; the representation of the task description is respectively passed through the matrix W Q ,W K ,W V Mapping, get the text vector Q1, (K t1 ,V t1 );

[0072] The text vector Q1, (K t1 ,V t1)、(K m ,V m ) inputs the first layer of the multimodal text encoder, and the first layer of the multimodal text encoder outputs representation; the first layer of the multimodal text encoder output representation is respectively passed through W Q ,W K ,W V Matrix mapping, get the text vector Q2, (K t2 ,V t2 );

[0073] The text vector Q2, (K t2 ,V t2 )、(K m ,V m ) inputs the second layer of the multimodal text encoder, and the second layer of the multimodal text encoder outputs representation; the second layer of the multimodal text encoder output representation is respectively passed through W Q ,W K ,W V Matrix mapping, get the text vector Q3, (K t3 ,V t3 );

[0074] The text vector Q3, (K t3 ,V t3 )、(K m ,V m ) input the third layer of the multimodal text encoder, the third layer output representation of the multimodal text encoder is used as the task-related molecular representation output by the multimodal text encoder, that is, the output of the dual-tower model;

[0075] 3) Compute the contrastive learning task loss based on the task-related molecular representation output by the multimodal text encoder and the universal molecular representation output by the molecular encoder

[0076] 4) Calculate the multi-regression prediction task loss based on the predicted value and true value output by the dual-tower model

[0077] The predicted value output by the dual-tower model is the output of the linear layer passing the task-related molecular representation output by the multimodal text encoder;

[0078] 5) Calculate the loss of contrastive learning task and multi-regression prediction task loss The sum of is used as the total loss function L of the twin tower model;

[0079] Introducing contrastive learning tasks to enhance the generalization ability of the model in classification tasks;

[0080] 6) The optimization of the twin tower model uses a momentum update mechanism:

[0081] Among them, θ t represents the twin-tower model parameters corresponding to the tth iteration, and m represents the momentum coefficient; Represents theta t Derivative; L represents the total loss function of the twin tower model; θ t+1 represents the parameters of the twin-tower model corresponding to the t+1th iteration;

[0082] 7) Repeat 1)-6) until the total loss function L of the dual-tower model converges to obtain a trained dual-tower model.

[0083] The other steps and parameters are the same as those in the first to third embodiments.

[0084] Specific embodiment 5: This embodiment differs from any one of specific embodiments 1 to 4 in that: 3) the contrastive learning task loss is calculated based on the task-related molecular representation output by the multimodal text encoder and the universal molecular representation output by the molecular encoder. The specific process is:

[0085] The task-related molecular representation h output by the multimodal text encoder t The mapping function f(·) with a nonlinear activation function is mapped to the universal molecular representation h output by the molecular encoder. m Same dimensions;

[0086] The text representation after mapping is denoted as h′ t =f(h t );

[0087] Represent the mapped text h′ t and molecular characterization m Perform L2 normalization and get and

[0088]

[0089] Where N is the number of samples, i is the i-th sample, and j is the j-th sample;

[0090] is the universal molecular representation output after the i-th sample passes through the molecular encoder and is normalized;

[0091] is the universal molecular representation output by the jth sample after passing through the molecular encoder and then normalized;

[0092] is the task-related molecular representation output by the i-th sample after passing through the multimodal text encoder and then normalized;

[0093] is the task-related molecular representation of the jth sample after passing through the multimodal text encoder and then normalized;

[0094] sim(,) is the cosine similarity, and τ represents the temperature coefficient of contrastive learning.

[0095] Other steps and parameters are the same as those in Specific Embodiments 1 to 4-1.

[0096] Specific embodiment 6: This embodiment differs from any one of specific embodiments 1 to 5 in that: the multi-regression prediction task loss The expression is:

[0097]

[0098] Where N is the number of samples, k is the kth attribute, and j is the jth sample;

[0099] M is the total number of attributes, M = 209,

[0100] m j ∈{0,1} M is a binary mask vector,

[0101] m kj =1 means that the kth attribute of the jth sample exists (valid);

[0102] m kj =0 means the kth attribute of the jth sample is missing;

[0103] y kj is the true value of the kth attribute of the jth sample;

[0104] is the predicted value of the kth attribute of the jth sample;

[0105] The predicted value is the task-related molecular representation output by the multimodal text encoder, which is mapped through a linear layer;

[0106] Only if the attribute exists (m kj =1), the corresponding error term will be counted as a loss.

[0107] Other steps and parameters are the same as those in Specific Implementations 1 to 5-1.

[0108] Specific embodiment 7: This embodiment differs from any one of specific embodiments 1 to 6 in that the representation of the output of the cross attention layer of the first layer in the multimodal text encoder is specifically:

[0109] (K m ,Vm ) and (K t1 ,V t1 ) are concatenated to obtain the cross-attention layer output representation of the first layer in the multimodal encoder; the expression is:

[0110]

[0111] Among them, Q is the text query vector, K is the key vector, and V is the value vector;

[0112] d k is the dimension value of vector Q and K;

[0113] The superscript T means to find the transpose;

[0114] Softmax() represents the activation function;

[0115] [K m ,K t1 ] is the concatenated key vector, [V m ,V t1 ] is the concatenated value vector.

[0116] The other steps and parameters are the same as those in the first to sixth embodiments.

[0117] Specific embodiment 8: This embodiment differs from any one of specific embodiments 1 to 7 in that the representation of the output of the cross attention layer of the second layer in the multimodal text encoder is specifically:

[0118] (K m ,V m ) and (K t2 ,V t2 ) are concatenated to obtain the cross-attention layer output representation of the second layer in the multimodal encoder; the expression is:

[0119]

[0120] The other steps and parameters are the same as those in the first to seventh embodiments.

[0121] Specific embodiment 9: This embodiment differs from any one of specific embodiments 1 to 8 in that the representation of the output of the cross attention layer of the third layer in the multimodal text encoder is specifically:

[0122] (K m ,V m ) and (K t3 ,V t3 ) for splicing; get the cross attention layer output representation of the third layer in the multimodal encoder; the expression is:

[0123]

[0124] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0125] Evaluate model performance: Use the linear probe method to evaluate the performance of task-enhanced molecular representations on classification and regression tasks on a benchmark test set. Use a large language model to analyze specific tasks and generate corresponding task descriptions. Input the task descriptions and molecules into the model to obtain task-related molecular representations, which are then predicted using a linear layer.

[0126] 1) Evaluate task-enhanced molecular representations using a linear probe approach on a benchmark test set;

[0127] MoleculeNet (WU Z, RAMSUNDAR B, FEINBERG EN, et al. Moleculenet: a benchmark for molecular machine learning [J]. Chemical science, 2018, 9(2): 513-530.) was used as the evaluation benchmark, and seven representative molecular property prediction tasks were selected to comprehensively evaluate the performance of molecular characterization. Specifically, they include:

[0128] Classification tasks: BBBP (blood-brain barrier permeability), ClinTox (clinical toxicity), HIV (anti-AIDS activity), Tox21 (toxicological endpoint);

[0129] Regression tasks: ESOL (water solubility), FreeSolv (hydration free energy), Lipophilicity (lipophilicity);

[0130] These tasks cover several key areas including physiology, biophysics and physical chemistry.

[0131] Linear probe method: This is one of the standard methods for evaluating the quality of characterization. The specific approach is:

[0132] Freeze the pre-trained model parameters

[0133] Add and train a linear classification / regression layer only on the representation output by the pre-trained model

[0134] In this way you can:

[0135] Controlling Assessment Variables

[0136] Directly reflects the discriminative ability of representation

[0137] Ensure fairness and comparability of assessments

[0138] Task-enhanced molecular characterization:

[0139] like Figure 2 As shown, this refers to the task-related molecular representation output by the multimodal text encoder

[0140] 2) Use GPT-4 to analyze specific tasks and generate corresponding task descriptions;

[0141] The tester / user uses GPT-4 to generate a task description.

[0142] The templates for interacting with GPT-4 are shown in the following table:

[0143]

[0144] The task description is that GPT-4 analyzes which features are task-related features that are helpful in solving the task and guides the model to adjust the molecular representation.

[0145] Example: As shown in the following table:

[0146]

[0147]

[0148] 3) Input the task description and molecules into the model to obtain the task-related molecular representation;

[0149] 4) Make predictions through the linear layer.

[0150] like Figure 1 The task-related representation output by the model is passed through the linear layer to predict the label of the task. For the regression task, the attribute value is predicted, such as molecular weight: 343.123, while for the regression task, the binary classification label is predicted through the sigmoid activation function.

[0151] The following examples are used to verify the beneficial effects of the present invention:

[0152] Compared with other methods, the following table shows the experimental results of the method of the present invention on classification and regression tasks. The experimental results show that the method of the present invention performs well in both classification and regression tasks, with average performance of 84.29 and 1.0317 on classification and regression tasks, respectively, exceeding the existing SOTA model. Specifically, the method of the present invention achieved the best performance on the ClinTox task, and achieved performance improvement compared to the base model (CHEM-BERT) used in all classification tasks. For regression tasks, the method of the present invention also achieved the best performance on the ESOL and FreeSolv tasks, and also achieved performance improvement compared to the base model on the Lip task. This shows that contrastive learning can not only improve the performance of classification tasks, but also help improve the performance of regression tasks. The experimental results demonstrate the effectiveness of the method of the present invention, which can enable the model to achieve stable improvement in all tasks.

[0153]

[0154]

[0155] The present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A molecular representation learning system enhanced with task description, characterized by: The system comprises: Multi-regression task dataset construction module based on task description, dual-tower model construction module, dual-tower model training module, and prediction module; The multi-regression task dataset construction module based on task description is used to construct the multi-regression task dataset MT-MTR based on task description. The specific process is as follows: First, obtain molecules from the DrugBank and ChEBI databases, and use RDKit to normalize the molecules in the DrugBank and ChEBI databases to obtain standardized SMILES format molecules; Deduplication is performed on the molecules in the normalized SMILES format to obtain the normalized SMILES format molecules after deduplication; 2. Get a list of molecular descriptors; The molecular descriptor list contains the names of each molecular descriptor and the functions for calculating each molecular descriptor; 3. For each molecule obtained in step 1, randomly select K descriptor names and calculation functions corresponding to the descriptor names from the molecular descriptor list obtained in step 2; Calculate the properties of each molecule using K randomly selected calculation functions; The K attributes of each molecule serve as K regression labels for the molecular descriptor; Input K randomly selected descriptor names into the large language model, and the large language model outputs a task description; Fourth, construct triplet data (molecule, task description, regression label) based on the first obtained molecule, the third obtained task description and the regression label, and construct the multi-regression task dataset MT-MTR based on the triplet data; The twin tower model building module is used to build a twin tower model; The dual-tower model training module is used to obtain the trained dual-tower model; The prediction module is used to input the molecules to be tested into the trained dual-tower model, and the trained dual-tower model outputs the task-related molecular representation; The twin tower model construction module is used to construct a twin tower model; the specific process is: The two-tower model includes a text encoder and a molecular encoder; The text encoder is the language pre-training model PubMedBERT; The molecular encoder is a molecular pre-training model CHEM-BERT; The language pre-training model PubMedBERT is divided into a unimodal text encoder and a multimodal text encoder; The unimodal text encoder is the 1st to 9th layers in the language pre-training model PubMedBERT; The multimodal text encoder is the 10th to 12th layers in the language pre-training model PubMedBERT; Change each self-attention layer in the multimodal text encoder to a cross-attention layer; The model training module is used to obtain the trained dual-tower model; the specific process is: 1) The molecules in the triplet data are input into the molecular encoder, and the molecular encoder outputs the universal molecular representation; the universal molecular representation is mapped through the linear layer to obtain a vector (K m ,V m ); 2) The task description in the triple data is input into the unimodal text encoder, and the unimodal text encoder outputs the representation of the task description; the representation of the task description is respectively passed through the matrix W Q ,W K ,W V Mapping, get the text vector Q1, (K t1 ,V t1 ); The text vector Q1, (K t1 ,V t1 )、(K m ,V m ) inputs the first layer of the multimodal text encoder, and the first layer of the multimodal text encoder outputs representation; the first layer of the multimodal text encoder output representation is respectively passed through W Q ,W K ,W V Matrix mapping, get the text vector Q2, (K t2 ,V t2 ); The text vector Q2, (K t2 ,V t2 )、(K m ,V m ) inputs the second layer of the multimodal text encoder, and the second layer of the multimodal text encoder outputs representation; the second layer of the multimodal text encoder output representation is respectively passed through W Q ,W K ,W V Matrix mapping, get the text vector Q3, (K t3 ,V t3 ); The text vector Q3, (K t3 ,V t3 )、(K m ,V m ) input the third layer of the multimodal text encoder, the third layer output representation of the multimodal text encoder is used as the task-related molecular representation output by the multimodal text encoder, that is, the output of the dual-tower model; 3) Compute the contrastive learning task loss based on the task-related molecular representation output by the multimodal text encoder and the universal molecular representation output by the molecular encoder 4) Calculate the multi-regression prediction task loss based on the predicted value and true value output by the dual-tower model The predicted value output by the dual-tower model is the output of the linear layer passing the task-related molecular representation output by the multimodal text encoder; 5) Calculate the loss of contrastive learning task and multi-regression prediction task loss The sum of is used as the total loss function L of the twin tower model; 6) The optimization of the twin tower model uses a momentum update mechanism: Among them, θ t represents the twin-tower model parameters corresponding to the tth iteration, and m represents the momentum coefficient; Represents theta t Derivative; L represents the total loss function of the twin tower model; θ t+1 represents the parameters of the twin-tower model corresponding to the t+1th iteration; 7) Repeat 1)-6) until the total loss function L of the dual-tower model converges to obtain a trained dual-tower model.

2. The molecular representation learning system with enhanced task description according to claim 1, characterized in that: The second step is to obtain a molecular descriptor list; the specific process is as follows: Use Descriptors in RDKit to get the list of all molecular descriptors supported by RDKit Descriptors.descList; The molecular descriptor list contains the name of each molecular descriptor and the function for calculating each molecular descriptor.

3. The molecular representation learning system with enhanced task description according to claim 2, characterized in that: 3) Compute the contrastive learning task loss based on the task-related molecular representation output by the multimodal text encoder and the universal molecular representation output by the molecular encoder The specific process is: The task-related molecular representation h output by the multimodal text encoder t The mapping function f(·) with a nonlinear activation function is mapped to the universal molecular representation h output by the molecular encoder. m Same dimensions; The text representation after mapping is denoted as h′ t =f(h t ); The mapped text representation h t ′ and molecular characterization h m Perform L2 normalization and get and Where N is the number of samples, i is the i-th sample, and j is the j-th sample; is the universal molecular representation output after the i-th sample passes through the molecular encoder and is normalized; is the universal molecular representation output by the jth sample after passing through the molecular encoder and then normalized; is the task-related molecular representation output by the i-th sample after passing through the multimodal text encoder and then normalized; is the task-related molecular representation of the jth sample after passing through the multimodal text encoder and then normalized; sim(,) is the cosine similarity, and τ represents the temperature coefficient of contrastive learning.

4. The molecular representation learning system with enhanced task description according to claim 3, characterized in that: The multi-regression prediction task loss The expression is: Where N is the number of samples, k is the kth attribute, and j is the jth sample; M is the total number of attributes, M = 209, m j ∈{0,1} M is a binary mask vector, m kj =1 means the kth attribute of the jth sample exists; m kj =0 means the kth attribute of the jth sample is missing; y kj is the true value of the kth attribute of the jth sample; is the predicted value of the kth attribute of the jth sample.

5. The molecular representation learning system with enhanced task description according to claim 4, characterized in that: The representation of the output of the cross-attention layer of the first layer in the multimodal text encoder is specifically: (K m ,V m ) and 9K t1 ,V t1 0 to obtain the cross-attention layer output representation of the first layer in the multimodal encoder; specifically: Among them, Q is the text query vector, K is the key vector, and V is the value vector; d k is the dimension value of vector Q and K; The superscript T means to find the transpose; Softmax() represents the activation function; [K m ,K t1 ] is the concatenated key vector, [V m ,V t1 ] is the concatenated value vector.

6. The molecular representation learning system with enhanced task description according to claim 5, characterized in that: The representation of the output of the cross-attention layer of the second layer in the multimodal text encoder is specifically: (K m ,V m ) and (K t2 ,V t2 ) are concatenated to obtain the cross-attention layer output representation of the second layer in the multimodal encoder; the expression is:

7. The molecular representation learning system with enhanced task description according to claim 6, characterized in that: The representation of the output of the cross-attention layer of the third layer in the multimodal text encoder is specifically: (K m ,V m ) and (K t3 ,V t3 ) for splicing; get the cross attention layer output representation of the third layer in the multimodal encoder; the expression is:

Images