Task description enhanced molecular characterization learning system

Abstract

The invention discloses a task description enhanced molecular characterization learning system, belongs to the field of molecular characterization learning in artificial intelligence in the field of computers, and particularly relates to the task description enhanced molecular characterization learning system. The method aims to solve the problems that in the prior art, general molecular characterization is difficult to capture task specific features, generalization performance is insufficient in a labeling data scarcity scene, molecular characterization learning accuracy is low, and interpretability is poor. The system comprises a task description-based multi-regression task data set construction module, a double-tower model construction module, a double-tower model training module and a prediction module. The task description-based multi-regression task data set construction module is used for constructing a task description-based multi-regression task data set; the double-tower model construction module is used for constructing a double-tower model; the double-tower model training module is used for acquiring a trained double-tower model; the prediction module is used for inputting molecules to be tested into the trained double-tower model, and the trained double-tower model outputs task-related molecular characterization.

Description

Technical Field

[0001] The present invention belongs to the field of molecular representation learning in artificial intelligence in the computer field, and specifically relates to a molecular representation learning system with enhanced task description. Background Art

[0002] With the rapid development of artificial intelligence technology, deep learning has been widely used in the fields of drug discovery and chemistry, especially in tasks such as compound screening and drug interaction prediction, where molecular representation learning has become an important research direction. The core of molecular representation is to convert the molecular structure into a mathematical vector representation so that it can be used by machine learning models to predict downstream tasks.

[0003] Traditional molecular characterization methods include molecular descriptors and molecular fingerprints constructed through expert knowledge, such as Morgan fingerprints (Morgan HL (1965) The generation of a unique machine description for chemical structures—a technique developed at chemical abstracts service. JChem Doc 5:107–113. https: / / doi.org / 10.1021 / c160017a018) and ECFP4 (Rogers D, Hahn M (2010) Extended-connectivity fingerprints. J Chem Inf Model 50:742–754. https: / / doi.org / 10.1021 / ci100050t). Although these methods have strong interpretability, they have obvious limitations in capturing complex molecular features. With the rise of deep learning, many studies have begun to use neural networks to learn molecular representations from labeled data (Gilmer J, Schoenholz SS, Riley PF, et al. Neural message passing for quantum chemistry [C] / / Proceedings of the 34th International Conference on Machine Learning. PMLR, 2017: 1263-1272.), (Yang K, Swanson K, Jin W, et al. Analyzing learned molecular representations for property prediction [J]. Journal of chemical information and modeling, 2019, 59 (8): 3370-3388.). These methods have shown significant advantages in specific tasks, but due to the scarcity of high-quality labeled data, the performance and generalization ability of the model are often difficult to break through the bottleneck.

[0004] In recent years, the success of self-supervised learning models, especially pre-trained language models in natural language processing, has triggered a research boom in applying pre-trained models to molecular representation learning (Devlin J, Chang MW, Lee K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding [C] / / Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019: 4171-4186.). Molecules are represented by sequences or graph structures and pre-trained using self-supervised tasks similar to language models. This pre-training paradigm significantly improves the representation ability and generalization of the model (Chithrananda S, Grand G, Ramsundar B. ChemBERTa: Large-scale self-supervised pretraining for molecular property prediction [J]. arXiv preprint arXiv: 2010.09885, 2020.), (Rong Y, Bian Y, Xu T, et al. Self-supervised graph transformer on large-scale molecular data [C] / / Advances in Neural Information Processing Systems. 2020, 33: 12559-12571.).At the same time, there are also some methods that introduce more external information (such as knowledge graphs (Fang X, Liu Y, Chen L, et al. Knowledge graph contrastive learning for molecular property prediction [C] / / Proceedings of the 2022 SIAM International Conference on Data Mining (SDM). SIAM, 2022: 244-252.) or text data (Zeng D, Zhang J, Li R, et al. KV-PLM: A knowledge-enhanced pre-trained language model for molecular property prediction [J]. arXiv preprint arXiv:2205.00464,2022.) to enhance molecular representation. Existing molecular representation learning methods have achieved remarkable results in generating universal molecular representations. Although universal molecular representations can be used for a variety of tasks, they may not always be the best fit for a specific task. This is because different molecular property prediction tasks often require attention to different molecular features. For example, toxicity prediction may require a focus on known toxic substructures, while binding affinity prediction requires special attention to structural features that interact with specific protein targets. Universal representations are unlikely to capture these task-specific detail features well.

[0005] In this context, the molecular representation learning method enhanced by task description shows unique advantages and important value. First, by introducing task description to assist molecular representation learning, the model can better distinguish between key features and minor features in a specific task. This task-aware ability is particularly important for scenarios where labeled data is scarce, and can significantly improve the generalization performance of the model. For example, in the drug-target interaction prediction task, the model can automatically adjust the focus on the binding site-related regions in the molecular structure based on the description information of the target. Secondly, task-related molecular representation has obvious advantages in computational efficiency and interpretability. Since molecular representation is optimized for specific tasks, the model can often achieve better performance with the same parameters and computing resources. At the same time, the embedded representation generated by this method has stronger interpretability and can establish a clear association between the specific structural features of the molecule and the task results. This is of great significance for understanding the reasons for the prediction results and guiding the optimization design of molecules.

[0006] In order to solve the above problems, the present invention proposes a new method, which combines the natural language understanding ability of the language model and uses text prompts to guide the generation of molecular representations, so that the model can adjust molecular features according to task requirements, thereby generating task-related molecular representations and improving the performance of downstream tasks. Summary of the invention

[0007] The purpose of the present invention is to solve the problems in the prior art that general molecular representations are difficult to capture task-specific features and have insufficient generalization performance in scenarios with scarce labeled data, resulting in low accuracy and poor interpretability in molecular representation learning, and to propose a molecular representation learning system with enhanced task description.

[0008] A task description enhanced molecular representation learning system includes:

[0009] Multi-regression task dataset construction module based on task description, dual-tower model construction module, dual-tower model training module, and prediction module;

[0010] The multi-regression task dataset construction module based on task description is used to construct the multi-regression task dataset MT-MTR based on task description; the specific process is as follows:

[0011] 1. Obtain molecules from DrugBank and ChEBI databases, and use RDKit to normalize the molecules in DrugBank and ChEBI databases to obtain standardized SMILES format molecules;

[0012] Deduplication is performed on the molecules in the normalized SMILES format to obtain the normalized molecules in the SMILES format after deduplication;

[0013] 2. Get a list of molecular descriptors;

[0014] The molecular descriptor list includes the name of each molecular descriptor and the function for calculating each molecular descriptor;

[0015] 3. For each molecule in the molecules obtained in step 1, randomly select K descriptor names and calculation functions corresponding to the descriptor names from the molecular descriptor list obtained in step 2;

[0016] Calculate the properties of each molecule using K randomly selected calculation functions;

[0017] The K attributes of each molecule serve as K regression labels for the molecular descriptor;

[0018] Input the randomly selected K descriptor names into the large language model, and the large language model outputs 1 task description;

[0019] Fourth, construct triple data (molecule, task description, regression label) based on the first obtained molecule, the third obtained task description and the regression label, and construct the multi-regression task dataset MT-MTR based on the triple data;

[0020] The twin tower model building module is used to build a twin tower model;

[0021] The dual-tower model training module is used to obtain the trained dual-tower model;

[0022] The prediction module is used to input the molecules to be tested into the trained double-tower model, and the trained double-tower model outputs task-related molecular representations.

[0023] The beneficial effects of the present invention are:

[0024] The present invention discloses a molecular representation learning method with enhanced task description based on a language pre-training model and a molecular pre-training model.

[0025] This method adjusts the molecular representation through textual prompts so that it can be better suited for specific downstream tasks, thereby improving prediction accuracy and training efficiency. This technology can be widely used in drug discovery, molecular property prediction and other fields.

[0026] This paper proposes a molecular representation learning method enhanced by task description, which realizes the task specificity enhancement of molecular representation by combining task description with molecular representation. This method not only injects the knowledge contained in the text, but also utilizes the reasoning ability of the language model, providing a new research idea for molecular representation learning.

[0027] 1. Aiming at the problem that general molecular representation in the prior art is difficult to capture task-specific features, the present invention introduces task description to guide the generation of molecular representation. Experimental results show that this method has achieved significant improvements in multiple regression tasks and achieved optimal performance on ESOL, FreeSolv and Lipophilicity datasets.

[0028] 2. To address the problem of insufficient generalization performance in scenarios with scarce labeled data, the present invention constructs a multi-regression task dataset MT-MTR based on task descriptions and designs corresponding pre-training tasks. Experimental verification shows that this method can not only focus on relevant features explicitly mentioned in the task description, but also identify unmentioned but relevant features, while reducing the focus on irrelevant features, demonstrating good generalization capabilities.

[0029] 3. To address the problem of poor interpretability, this paper verifies that the model does pay attention to key information related to the task by analyzing the attention matrix. For example, in the solubility prediction task, the model focuses on tokens such as "groups" and "ethers" that are closely related to solubility, as well as polar functional groups in molecules.

[0030] 4. The present invention has good transferability and has achieved similar performance improvements on different molecular pre-training models (such as ChemBERTa-2 and CHEM-BERT). This shows that the method can inherit the advantages of different basic models and has broad application prospects. The method can be applied to molecular property prediction tasks in the fields of drug discovery, material design, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart of the overall scheme of the system of the present invention;

[0032] Figure 2 This is a system model diagram of the present invention. DETAILED DESCRIPTION

[0033] Specific implementation method 1: In this implementation method, a molecular representation learning system with enhanced task description includes:

[0034] Multi-regression task dataset construction module based on task description, dual-tower model construction module, dual-tower model training module, and prediction module;

[0035] The multi-regression task dataset construction module based on task description is used to construct the multi-regression task dataset MT-MTR based on task description; the specific process is as follows:

[0036] 1. Obtain molecules from the DrugBank (Knox C, Wilson M, Klinger CM, et al. DrugBank 6.0: the DrugBank knowledgebase for 2024 [J]. Nucleic acids research, 2024, 52 (D1): D1265-D1275.) and ChEBI (Hastings J, Owen G, Dekker A, et al. ChEBI in 2016: Improved services and an expanding collection of metabolites [J]. Nucleic acids research, 2016, 44 (D1): D1214-D1219.) databases, and use RDKit to standardize the molecules in the DrugBank and ChEBI databases to obtain standardized SMILES format molecules;

[0037] Deduplication is performed on the molecules in the normalized SMILES format to obtain the normalized molecules in the SMILES format after deduplication;

[0038] 2. Get a list of molecular descriptors;

[0039] The molecular descriptor list includes the name of each molecular descriptor and the function for calculating each molecular descriptor;

[0040] 3. For each molecule in the molecules obtained in step 1, randomly select K (5-10) descriptor names and calculation functions corresponding to the descriptor names from the molecular descriptor list obtained in step 2;

[0041] Calculate the properties of each molecule using K randomly selected calculation functions;

[0042] The K attributes of each molecule serve as K regression labels for the molecular descriptor;

[0043] Input the randomly selected K descriptor names into the large language model, and the large language model outputs 1 task description;

[0044] The regression label consists of two 209-dimensional vectors. The first vector stores the value of the regression label, and the second vector consists of 0 and 1, where 0 indicates missing and 1 indicates the presence of the regression label. There are K 1s in the second vector.

[0045] Fourth, construct triple data (molecule, task description, regression label) based on the first obtained molecule, the third obtained task description and the regression label, and construct the multi-regression task dataset MT-MTR based on the triple data;

[0046] The twin tower model building module is used to build a twin tower model;

[0047] The dual-tower model training module is used to obtain the trained dual-tower model;

[0048] The prediction module is used to input the molecules to be tested into the trained double-tower model, and the trained double-tower model outputs task-related molecular representations.

[0049] Specific implementation method 2: This implementation method is different from the specific implementation method 1 in that: the second embodiment obtains a molecular descriptor list; the specific process is:

[0050] Use Descriptors in RDKit to get a list of all molecular descriptors supported by RDKit

[0051] Descriptors.descList;

[0052] The molecular descriptor list contains 209 molecular descriptor names and functions for calculating each molecular descriptor, including but not limited to the following features: maximum absolute electronic state index (MaxAbsEStateIndex), maximum electronic state index (MaxEStateIndex), minimum absolute electronic state index (MinAbsEStateIndex), minimum electronic state index (MinEStateIndex), quantitative drug similarity score (QED, Quantitative Estimate ofDrug-likeness), molecular weight (MolWt, Molecular Weight);

[0053] 1) Molecular data were obtained from DrugBank and ChEBI databases, and RDKit was used to remove duplicates and standardize them, resulting in 55,759 valid SMILES;

[0054] 2) Calculate 209 molecular properties of each molecule using RDKit;

[0055] 3) Randomly sample 5-10 properties for each molecule, generate task descriptions for the sampled property names through GPT-3.5, and use the corresponding property values ​​as regression labels;

[0056] 4) Finally construct a (molecule, task description, regression label) triplet dataset.

[0057] The other steps and parameters are the same as those in the first embodiment.

[0058] Specific implementation method three: This implementation method is different from specific implementation methods one or two in that: the double tower model construction module is used to construct a double tower model; the specific process is:

[0059] The dual-tower model includes a text encoder (T-Encoder) and a molecular encoder (M-Encoder);

[0060] The text encoder (T-Encoder) is a language pre-training model PubMedBERT;

[0061] The molecular encoder (M-Encoder) is a molecular pre-training model CHEM-BERT;

[0062] The language pre-training model PubMedBERT is divided into a unimodal text encoder and a multimodal text encoder;

[0063] The unimodal text encoder is the 1st to 9th layers in the language pre-training model PubMedBERT;

[0064] The multimodal text encoder is the 10th to 12th layers in the language pre-training model PubMedBERT;

[0065] Each self-attention layer in the multimodal text encoder is changed to a cross-attention layer to achieve the fusion of task description embedding and molecular representation.

[0066] The unimodal encoder is used to understand the semantic information of the task description, and the multimodal encoder is used to adjust the molecular representation.

[0067] The other steps and parameters are the same as those in the first or second embodiment.

[0068] Specific implementation method 4: This implementation method is different from the specific implementation methods 1 to 3 in that:

[0069] The model training module is used to obtain the trained double-tower model; the specific process is:

[0070] 1) The molecules in the triplet data are input into the molecular encoder, and the molecular encoder outputs the universal molecular representation; the universal molecular representation is mapped through the linear layer to obtain a vector (K m ,V m );

[0071] 2) The task description in the triple data is input into the unimodal text encoder, and the unimodal text encoder outputs the representation of the task description; the representation of the task description is respectively passed through the matrix W Q ,W K ,W V Mapping, get the text vector Q1, (K t1 ,V t1 );

[0072] The text vector Q1, (K t1 ,V t1)、(K m ,V m ) is input into the first layer of the multimodal text encoder, and the first layer of the multimodal text encoder outputs the representation; the first layer of the multimodal text encoder output representation is respectively passed through W Q ,W K ,W V Matrix mapping, get the text vector Q2, (K t2 ,V t2 );

[0073] The text vector Q2, (K t2 ,V t2 )、(K m ,V m ) is input into the second layer of the multimodal text encoder, and the second layer of the multimodal text encoder outputs representation; the second layer output representation of the multimodal text encoder is respectively passed through W Q ,W K ,W V Matrix mapping, get the text vector Q3, (K t3 ,V t3 );

[0074] The text vector Q3, (K t3 ,V t3 )、(K m ,V m ) input the third layer of the multimodal text encoder, the third layer output representation of the multimodal text encoder is used as the task-related molecular representation output by the multimodal text encoder, that is, the output of the dual tower model;

[0075] 3) Compute the contrastive learning task loss based on the task-related molecular representation output by the multimodal text encoder and the universal molecular representation output by the molecular encoder

[0076] 4) Calculate the multi-regression prediction task loss based on the predicted value and true value output by the dual-tower model

[0077] The prediction value output by the dual-tower model is the output of the linear layer passing the task-related molecular representation output by the multimodal text encoder;

[0078] 5) Calculate the loss of contrastive learning task and multi-regression prediction task loss The sum of is used as the total loss function L of the twin tower model;

[0079] Introducing contrastive learning tasks to enhance the generalization ability of the model in classification tasks;

[0080] 6) The optimization of the twin tower model uses a momentum update mechanism:

[0081] Among them, θ t represents the double tower model parameters corresponding to the tth iteration, and m represents the momentum coefficient; Represents the t Derivative; L represents the total loss function of the twin tower model; θ t+1 represents the parameters of the double tower model corresponding to the t+1th iteration;

[0082] 7) Repeat 1)-6) until the total loss function L of the dual-tower model converges to obtain a trained dual-tower model.

[0083] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0084] Specific implementation method 5: This implementation method is different from any one of the specific implementation methods 1 to 4 in that: 3) Computing the contrastive learning task loss based on the task-related molecular representation output by the multimodal text encoder and the universal molecular representation output by the molecular encoder The specific process is:

[0085] The task-related molecular representation h output by the multimodal text encoder t Through the mapping function f(·) with nonlinear activation function, it is mapped to the universal molecular representation h output by the molecular encoder. m Same dimensions;

[0086] The text representation after mapping is denoted as h′ t =f(h t );

[0087] Represent the mapped text h′ t and molecular characterization m Perform L2 normalization and get and

[0088]

[0089] Where N is the number of samples, i is the i-th sample, and j is the j-th sample;

[0090] is the universal molecular representation output after the i-th sample passes through the molecular encoder and is normalized;

[0091] is the universal molecular representation output by the jth sample after passing through the molecular encoder and then normalized;

[0092] is the task-related molecular representation of the i-th sample after passing through the multimodal text encoder and then normalized;

[0093] is the task-related molecular representation of the jth sample after passing through the multimodal text encoder and then normalized;

[0094] sim(,) is the cosine similarity, and τ represents the temperature coefficient of contrastive learning.

[0095] The other steps and parameters are the same as those in Specific Implementation 1 to 4-1.

[0096] Specific implementation method 6: This implementation method is different from any one of the specific implementation methods 1 to 5 in that: the multi-regression prediction task loss The expression is:

[0097]

[0098] Where N is the number of samples, k is the kth attribute, and j is the jth sample;

[0099] M is the total number of attributes, M = 209,

[0100] m j ∈{0,1} M is a binary mask vector,

[0101] m kj =1 means that the kth attribute of the jth sample exists (valid);

[0102] m kj =0 means the kth attribute of the jth sample is missing;

[0103] y kj is the true value of the kth attribute of the jth sample;

[0104] is the predicted value of the kth attribute of the jth sample;

[0105] The predicted value is the task-related molecular representation output by the multimodal text encoder, which is mapped through a linear layer;

[0106] Only if the attribute exists (m kj =1), the corresponding error term will be counted as a loss.

[0107] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5-1.

[0108] Specific implementation method 7: This implementation method is different from any one of the specific implementation methods 1 to 6 in that: the representation of the output of the cross attention layer of the first layer in the multimodal text encoder is specifically:

[0109] (K m ,Vm ) and (K t1 ,V t1 ) are concatenated to obtain the cross-attention layer output representation of the first layer in the multimodal encoder; the expression is:

[0110]

[0111] Among them, Q is the text query vector, K is the key vector, and V is the value vector;

[0112] d k is the dimension value of vectors Q and K;

[0113] The superscript T means to find the transpose;

[0114] Softmax() represents the activation function;

[0115] [K m ,K t1 ] is the concatenated key vector, [V m ,V t1 ] is the concatenated value vector.

[0116] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0117] Specific implementation eight: This implementation differs from any one of specific implementations one to seven in that: the representation of the cross attention layer output of the second layer in the multimodal text encoder is specifically:

[0118] (K m ,V m ) and (K t2 ,V t2 ) are concatenated to obtain the cross-attention layer output representation of the second layer in the multimodal encoder; the expression is:

[0119]

[0120] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0121] Specific implementation method 9: This implementation method is different from any one of specific implementation methods 1 to 8 in that: the representation of the output of the cross attention layer of the third layer in the multimodal text encoder is specifically:

[0122] (K m ,V m ) and (K t3 ,V t3 ) are concatenated to obtain the output representation of the cross attention layer of the third layer in the multimodal encoder; the expression is:

[0123]

[0124] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0125] Evaluate model performance: Use the linear probe method to evaluate the performance of task-enhanced molecular representations in classification and regression tasks on the benchmark test set; use a large language model to analyze specific tasks and generate corresponding task descriptions, input the task descriptions and molecules into the model to obtain task-related molecular representations, and make predictions through the linear layer.

[0126] 1) Evaluate task-enhanced molecular representation using a linear probe approach on a benchmark test set;

[0127] MoleculeNet (WU Z, RAMSUNDAR B, FEINBERG EN, et al. Moleculenet: a benchmark for molecular machine learning [J]. Chemical science, 2018, 9 (2): 513-530.) was used as the evaluation benchmark, and 7 representative molecular property prediction tasks were selected to comprehensively evaluate the performance of molecular characterization. Specifically, they include:

[0128] Classification tasks: BBBP (blood-brain barrier permeability), ClinTox (clinical toxicity), HIV (anti-AIDS activity), Tox21 (toxicological endpoints);

[0129] Regression tasks: ESOL (water solubility), FreeSolv (hydration free energy), Lipophilicity (lipophilicity);

[0130] These tasks cover several key areas including physiology, biophysics and physical chemistry.

[0131] Linear probe method: This is one of the standard methods for evaluating the quality of characterization. The specific approach is:

[0132] Freeze the pre-trained model parameters

[0133] Add and train a linear classification / regression layer only on the representation output by the pre-trained model

[0134] In this way you can:

[0135] Controlling Evaluation Variables

[0136] Directly reflects the discriminative ability of representation

[0137] Ensure fairness and comparability of assessments

[0138] Mission-enhanced molecular characterization:

[0139] like Figure 2 As shown in Figure 2, this refers to the task-dependent molecular representations output by the multimodal text encoder.

[0140] 2) Use GPT-4 to analyze specific tasks and generate corresponding task descriptions;

[0141] The tester / user generates a task description using GPT-4.

[0142] The templates that interact with GPT-4 are shown in the following table:

[0143]

[0144] The task description is that GPT-4 analyzes which features are task-related features, which are helpful in solving the task, and guides the model to adjust the molecular representation.

[0145] Example: As shown in the following table:

[0146]

[0147]

[0148] 3) Input the task description and molecules into the model to obtain the task-related molecular representation;

[0149] 4) Make predictions through the linear layer.

[0150] like Figure 1 The task-related representation output by the model is passed through a linear layer to predict the label of the task. For regression tasks, the attribute value is predicted, such as molecular weight: 343.123, while for regression tasks, the binary classification label is predicted through a sigmoid activation function.

[0151] The following examples are used to verify the beneficial effects of the present invention:

[0152] Compared with other methods, the following table shows the experimental results of the method of the present invention on classification and regression tasks. The experimental results show that the method of the present invention performs well in both classification and regression tasks, and its average performance on classification and regression tasks is 84.29 and 1.0317 respectively, which exceeds the existing SOTA model. Specifically, the method of the present invention achieved the best performance on the ClinTox task, and achieved performance improvement compared to the base model (CHEM-BERT) used in all classification tasks. For regression tasks, the method of the present invention also achieved the best performance on the ESOL and FreeSolv tasks, and also achieved performance improvement on the Lip task compared to the base model. This shows that contrastive learning can not only improve the performance of classification tasks, but also help improve the performance of regression tasks. The experimental results prove the effectiveness of the method of the present invention, which can enable the model to achieve stable improvement in all tasks.

[0153]

[0154]

[0155] The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A molecular representation learning system with enhanced task description, characterized in that: The system comprises: Multi-regression task dataset construction module based on task description, dual-tower model construction module, dual-tower model training module, and prediction module; The multi-regression task dataset construction module based on task description is used to construct the multi-regression task dataset MT-MTR based on task description; the specific process is as follows:

1. Obtain molecules from DrugBank and ChEBI databases, and use RDKit to normalize the molecules in DrugBank and ChEBI databases to obtain standardized SMILES format molecules; Deduplication is performed on the molecules in the normalized SMILES format to obtain the normalized molecules in the SMILES format after deduplication; 2. Get a list of molecular descriptors; The molecular descriptor list includes the name of each molecular descriptor and the function for calculating each molecular descriptor; 3. For each molecule in the molecules obtained in step 1, randomly select K descriptor names and calculation functions corresponding to the descriptor names from the molecular descriptor list obtained in step 2; Calculate the properties of each molecule using K randomly selected calculation functions; The K attributes of each molecule serve as K regression labels for the molecular descriptor; Input the randomly selected K descriptor names into the large language model, and the large language model outputs 1 task description; Fourth, based on the obtained molecule, the obtained task description and the regression label, triplet data (molecule, task description, regression label) is constructed, and based on the triplet data, a multi-regression task dataset MT-MTR based on the task description is constructed. The twin tower model building module is used to build a twin tower model; The dual-tower model training module is used to obtain the trained dual-tower model; The prediction module is used to input the molecules to be tested into the trained double-tower model, and the trained double-tower model outputs task-related molecular representations.

2. The molecular representation learning system with enhanced task description according to claim 1, characterized in that: Second, obtain the molecular descriptor list; the specific process is: Use Descriptors in RDKit to get a list of all molecular descriptors supported by RDKit Descriptors.descList; The molecular descriptor list contains the name of each molecular descriptor and the function that calculates each molecular descriptor.

3. The molecular representation learning system with enhanced task description according to claim 2, characterized in that: The twin tower model construction module is used to construct a twin tower model; the specific process is: The two-tower model includes a text encoder and a molecular encoder; The text encoder is a language pre-training model PubMedBERT; The molecular encoder is a molecular pre-training model CHEM-BERT; The language pre-training model PubMedBERT is divided into a unimodal text encoder and a multimodal text encoder; The unimodal text encoder is the 1st to 9th layers in the language pre-training model PubMedBERT; The multimodal text encoder is the 10th to 12th layers in the language pre-training model PubMedBERT; Each self-attention layer in the multimodal text encoder is changed to a cross-attention layer.

4. The molecular representation learning system with enhanced task description according to claim 3, characterized in that: The model training module is used to obtain the trained double-tower model; the specific process is: 1) The molecules in the triplet data are input into the molecular encoder, and the molecular encoder outputs the universal molecular representation; the universal molecular representation is mapped through the linear layer to obtain a vector (K m ,V m ); 2) The task description in the triple data is input into the unimodal text encoder, and the unimodal text encoder outputs the representation of the task description; the representation of the task description is respectively passed through the matrix W Q ,W K ,W V Mapping, get the text vector Q1, (K t1 ,V t1 ); The text vector Q1, (K t1 ,V t1 )、(K m ,V m ) is input into the first layer of the multimodal text encoder, and the first layer of the multimodal text encoder outputs the representation; the first layer of the multimodal text encoder output representation is respectively passed through W Q ,W K ,W V Matrix mapping, get the text vector Q2, (K t2 ,V t2 ); The text vector Q2, (K t2 ,V t2 )、(K m ,V m ) is input into the second layer of the multimodal text encoder, and the second layer of the multimodal text encoder outputs representation; the second layer output representation of the multimodal text encoder is respectively passed through W Q ,W K ,W V Matrix mapping, get the text vector Q3, (K t3 ,V t3 ); The text vector Q3, (K t3 ,V t3 )、(K m ,V m ) input the third layer of the multimodal text encoder, the third layer output representation of the multimodal text encoder, as the task-related molecular representation output by the multimodal text encoder, that is, the output of the dual-tower model; 3) Compute the contrastive learning task loss based on the task-related molecular representation output by the multimodal text encoder and the universal molecular representation output by the molecular encoder 4) Calculate the multi-regression prediction task loss based on the predicted value and true value output by the dual-tower model The prediction value output by the dual-tower model is the output of the linear layer passing the task-related molecular representation output by the multimodal text encoder; 5) Calculate the loss of contrastive learning task and multi-regression prediction task loss The sum of is used as the total loss function L of the twin tower model; 6) The optimization of the twin tower model uses a momentum update mechanism: Among them, θ t represents the double tower model parameters corresponding to the tth iteration, and m represents the momentum coefficient; Represents the t Derivative; L represents the total loss function of the twin tower model; θ t+1 represents the parameters of the double tower model corresponding to the t+1th iteration; 7) Repeat 1)-6) until the total loss function L of the dual-tower model converges to obtain a trained dual-tower model.

5. The molecular representation learning system with enhanced task description according to claim 4, characterized in that: 3) Compute the contrastive learning task loss based on the task-related molecular representation output by the multimodal text encoder and the universal molecular representation output by the molecular encoder The specific process is: The task-related molecular representation h output by the multimodal text encoder t Through the mapping function f(·) with nonlinear activation function, it is mapped to the universal molecular representation h output by the molecular encoder. m Same dimensions; The mapped text representation is denoted as h t ′=f(h t ); The mapped text representation h t ′ and molecular characterization h m Perform L2 normalization and get and Where N is the number of samples, i is the i-th sample, and j is the j-th sample; is the universal molecular representation output after the i-th sample passes through the molecular encoder and is normalized; is the universal molecular representation output by the jth sample after passing through the molecular encoder and then normalized; is the task-related molecular representation of the i-th sample after passing through the multimodal text encoder and then normalized; is the task-related molecular representation of the jth sample after passing through the multimodal text encoder and then normalized; sim(,) is the cosine similarity, and τ represents the temperature coefficient of contrastive learning.

6. The molecular representation learning system with enhanced task description according to claim 5, characterized in that: The multi-regression prediction task loss The expression is: Where N is the number of samples, k is the kth attribute, and j is the jth sample; M is the total number of attributes, M = 209, m j ∈{0,1} M is a binary mask vector, m kj =1 means the kth attribute of the jth sample exists; m kj =0 means the kth attribute of the jth sample is missing; y kj is the true value of the kth attribute of the jth sample; is the predicted value of the kth attribute of the jth sample.

7. The molecular representation learning system with enhanced task description according to claim 6, characterized in that: The representation of the output of the cross-attention layer of the first layer in the multimodal text encoder is specifically: (K m ,V m ) and (K t1 ,V t1 ) to obtain the cross-attention layer output representation of the first layer in the multimodal encoder; specifically: Among them, Q is the text query vector, K is the key vector, and V is the value vector; d k is the dimension value of vectors Q and K; The superscript T means to find the transpose; Softmax() represents the activation function; [K m ,K t1 ] is the concatenated key vector, [V m ,V t1 ] is the concatenated value vector.

8. The molecular representation learning system with enhanced task description according to claim 7, characterized in that: The representation of the output of the cross-attention layer of the second layer in the multimodal text encoder is specifically: (K m ,V m ) and (K t2 ,V t2 ) are concatenated to obtain the cross-attention layer output representation of the second layer in the multimodal encoder; the expression is:

9. The molecular representation learning system with enhanced task description according to claim 8, characterized in that: The representation of the output of the cross-attention layer of the third layer in the multimodal text encoder is specifically: (K m ,V m ) and (K t3 ,V t3 ) are concatenated to obtain the output representation of the cross attention layer of the third layer in the multimodal encoder; The expression is:

Images