Molecular property prediction fine tuning method based on drug pre-training model

Abstract

The invention discloses a molecular property prediction fine tuning method based on a drug pre-training model, and belongs to the technical field of drug discovery. According to the method, a head feature parallel adapter is introduced into a pre-training model, and after a trunk part of the pre-training model is frozen, a trained specific molecular property prediction model is obtained by training the head feature parallel adapter and a drug molecular property prediction network; the head feature parallel adapter realizes deep learning of a complex chemical space in molecules by segmenting, adapting and merging molecular feature vectors, is composed of a full connection layer and a nonlinear activation function, focuses on optimizing molecular feature representation and keeps the efficiency of model parameters at the same time, not only improves the performance of a model, but also improves the performance of the model. The adapter has the advantages of being simple in structure and convenient to use, quickening the progress of drug discovery, reducing experiment cost, being wide in applicability, being capable of being popularized to a plurality of fields such as material science and environmental science, and providing an efficient and flexible tool for research in related fields.

Description

Technical Field

[0001] The invention relates to a molecular property prediction fine-tuning method based on a drug pre-training model, and belongs to the technical field of drug discovery. Background Art

[0002] Drug discovery is a complex and resource-intensive process. Traditional methods rely on experiments and clinical trials, which are time-consuming and costly. With the development of artificial intelligence technology, especially the application of deep learning in the field of bioinformatics, new possibilities have been brought to drug discovery. Pre-trained models, such as Graph Neural Networks, Graph Transformer Networks, and BERT, have shown great potential in drug discovery. They are able to obtain a comprehensive representation of molecular structure through generalization learning on unlabeled datasets. These models are able to predict a variety of molecular properties from quantum chemistry to physiology and are used as feature encoders.

[0003] However, these large networks are challenging to fine-tune for downstream tasks because they require storing and deploying separate copies of model parameters for each task, which is not only costly but often impractical. To overcome these limitations, researchers have proposed parameter-efficient fine-tuning (PEFT) methods. These methods freeze the background network and may introduce additional trainable parameters, or specify a set of parameters for training, or redefine the adaptation process to improve efficiency. Although these PEFT methods have performed well in natural language processing and computer vision tasks, their application in the field of drug discovery has been rarely studied.

[0004] Furthermore, existing PEFT methods face challenges in drug discovery applications because they adopt a one-size-fits-all strategy to incorporate uniform parameters across all downstream knowledge. While this uniformity simplifies the fine-tuning process, it ignores the multifaceted knowledge inherent in the pre-trained models and the multi-factor requirements of specific downstream tasks, resulting in weak models in explicitly learning the complex chemical space within molecules. In drug discovery, tasks requiring comprehensive knowledge are very common, and strong interpretability is critical in this field. Currently, few studies have focused on enhancing the interpretability and predictive power of models by learning various aspects of prior knowledge and downstream information during the fine-tuning process, which is a challenging task. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a molecular property prediction fine-tuning method based on a drug pre-training model, which is particularly suitable for the field of drug discovery. By designing a highly interpretable and high-performance fine-tuning paradigm, namely, the Head Feature Parallel Adapter (HFPA), effective fine-tuning of the pre-training model can be achieved. The technical solution is as follows:

[0006] The molecular property prediction fine-tuning method of the present invention comprises: introducing a head feature parallel adapter into a pre-trained model for drug molecular property prediction, freezing the main part of the pre-trained model, and obtaining a trained specific molecular property prediction model by training the head feature parallel adapter and a drug molecular property prediction network;

[0007] The calculation process of the head feature parallel adapter includes:

[0008] First, the feature extraction part of the pre-trained model is used to extract the feature vector x of the drug molecule. i , the characteristic vector x of the drug molecule i Decompose into h head features H of equal size ik , and at the same time, the feature vector x is segmented i Mapping to multiple subspaces:

[0009] Split(x i )=(Hi1,Hi2,…,Hih)k=1,2,…,h

[0010] Each head feature H ik Through an Adapter ik For processing, the adapter ik It includes: a down-projection layer, a nonlinear activation function and an up-projection layer; the down-projection layer maps the head feature to a lower dimension to obtain a low-dimensional feature x down , and then the nonlinear expression ability of the model is increased by the nonlinear activation function. Finally, the upper projection layer maps the feature back to the original dimension to obtain the original dimension feature x up ;

[0011] The Adapter ik The output H′ ik for:

[0012] Adapter(x)=Relu(W down x+b down )W up +b up

[0013] H′ik =Adapter ik (H ik )k=1,2,3,…,h

[0014] in, represents the downsampling matrix, represents the downsampling bias, represents the upsampling matrix, Represents the upsampling bias; M represents the dimension of the input x, R represents the bottleneck dimension, R<<M;

[0015] Head characteristics H′ after all adapters have processed ik The drug molecule feature vector x is merged by the concatenation operation and i Combined through residual connections, the fine-tuned output feature x′ is formed i :

[0016] x′ i =Concat(H′ i1 ,H′ i2 ,…,H′ ih )+x i

[0017] The fine-tuned output feature x′ i Input the trained drug molecular property prediction network to obtain the drug molecular properties.

[0018] Optionally, the method further includes: introducing a head feature parallel cross adapter HFPCA, where all low-dimensional adapter output features are first concatenated for joint learning Then through a fully connected layer is processed to learn the interaction between head features; finally, the segmentation operation Match the processed cross features to the number and dimensions of the input features.

[0019] Optionally, the method further includes: converting the structural information of the drug molecule into a SMILES string or other encoding form, and converting it into a numerical vector representation through a cheminformatics method.

[0020] Optionally, the method adopts a Transformer model, and starting from the second layer of the Transformer model, the head feature parallel adapter is inserted between the feedforward network of each Transformer layer and the subsequent layer normalization.

[0021] The present invention provides a molecular property prediction fine-tuning system, wherein the system introduces a head feature parallel adapter into a pre-trained model for drug molecular property prediction, freezes the main part of the pre-trained model, and then obtains a trained specific molecular property prediction model by training the head feature parallel adapter and a drug molecular property prediction network;

[0022] The head feature parallel adapter includes: a head feature segmentation module, a head feature learning module, and a head feature merging module;

[0023] The head feature segmentation module is used to extract the feature vector x of the drug molecule from the pre-training model. i Decompose into h head features H of equal size ik , and at the same time, the feature vector x is segmented i Mapping to multiple subspaces:

[0024] Split(x i )=(H i1 ,H i2 ,…,H ih )k=1,2,…,h

[0025] The head feature learning module performs a ik Through an Adapter ik For processing, the adapter ik It includes: a down-projection layer, a nonlinear activation function and an up-projection layer; the down-projection layer maps the head feature to a lower dimension to obtain a low-dimensional feature x down , and then the nonlinear expression ability of the model is increased by the nonlinear activation function. Finally, the upper projection layer maps the feature back to the original dimension to obtain the original dimension feature x up ;

[0026] The Adapter ik The output H′ ik for:

[0027] Adapter(x)=Relu(W down x+b down )W up +b up

[0028] H′ ik =Adapter ik (H ik )k=1,2,3,…,h

[0029] in, represents the downsampling matrix, represents the downsampling bias, represents the upsampling matrix, Represents the upsampling bias; M represents the dimension of the input x, R represents the bottleneck dimension, R<<M;

[0030] The head feature merging module combines all the head features H′ processed by the adapter ik The drug molecule feature vector x is merged by the concatenation operation and i Combined through residual connections, the fine-tuned output feature x′ is formed i :

[0031] x′ i =Concat(H′ i1 ,H′ i2 ,…,H′ ih )+x i

[0032] The fine-tuned output feature x′ i Input the trained drug molecular property prediction network to obtain the drug molecular properties.

[0033] Optionally, the system further comprises a head feature parallel cross adapter HFPCA, where all low-dimensional adapter output features are first concatenated for joint learning Then through a fully connected layer is processed to learn the interaction between head features; finally, the segmentation operation Match the processed cross features to the number and dimensions of the input features.

[0034] Optionally, the system converts the structural information of the drug molecule into a SMILES string or other encoding form, and converts it into a numerical vector representation through a cheminformatics method.

[0035] Optionally, the system adopts a Transformer model, and starting from the second layer of the Transformer model, the head feature parallel adapter is inserted between the feedforward network of each Transformer layer and the subsequent layer normalization.

[0036] The present invention provides an electronic device, comprising a memory and a processor;

[0037] The memory is used to store computer programs;

[0038] The processor is used to implement the molecular property prediction and fine-tuning method as described in any one of the above items when executing the computer program.

[0039] The present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the molecular property prediction and fine-tuning method as described in any one of the above items is implemented.

[0040] The beneficial effects of the present invention are:

[0041] The molecular property prediction fine-tuning method provided by the present invention introduces a head feature parallel adapter (HFPA) and its extended head feature parallel cross adapter (HFPCA) in the pre-trained model for drug molecular property prediction, and provides a sophisticated fine-tuning mechanism, which significantly improves the efficiency of drug discovery tasks and the adaptability of the model, while reducing the demand for computing resources.

[0042] In addition, the molecular property prediction fine-tuning method based on HFPA of the present invention effectively improves the accuracy of molecular property prediction by optimizing feature representation and enhancing the interpretability of the model, enabling the model to have a deeper understanding of the physicochemical properties of drug molecules; experimental results show that by reducing model parameters and promoting interactions between features, HFPA and HFPCA not only improve the performance of the model, but also accelerate the process of drug discovery and reduce experimental costs, and due to their wide applicability, these adapters can be extended to multiple fields such as materials science and environmental science, providing an efficient and flexible tool for research in related fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 A schematic diagram of the basic flow of a molecular property prediction and fine-tuning method provided in one embodiment of the present invention.

[0045] Figure 2 A schematic diagram of the position of a head feature parallel adapter embedded in a Transformer model provided by an embodiment of the present invention.

[0046] Figure 3 A schematic diagram of the structure of a head feature parallel adapter provided for one embodiment of the present invention.

[0047] Figure 4 A schematic diagram of the reasons for the explainability of the molecular property prediction fine-tuning method provided in one embodiment of the present invention.

[0048] Figure 5A schematic diagram of the interpretability analysis of the head features of the molecular property prediction fine-tuning method provided in one embodiment of the present invention.

[0049] Fig. 6A This is a diagram showing substantial changes in model performance caused by hyperparameter selection in one embodiment of the present invention.

[0050] Figure 6B This is a diagram analyzing the importance of hyperparameter selection on AUROC in one embodiment of the present invention.

[0051] Figure 6C This is a graph showing the relationship between hyperparameter selection and AUROC in one embodiment of the present invention.

[0052] Figure 7 A schematic diagram of the chemical feature space quality of a molecular property prediction fine-tuning method in one embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0054] Embodiment 1:

[0055] This embodiment provides a method for predicting drug molecular properties, including: introducing a head feature parallel adapter into a pre-trained model for drug molecular property prediction, freezing the main part of the pre-trained model, and obtaining a trained specific molecular property prediction model by training the head feature parallel adapter and a drug molecular property prediction network;

[0056] The calculation process of the head feature parallel adapter includes:

[0057] First, the feature extraction part of the pre-trained model is used to extract the feature vector x of the drug molecule. i , the characteristic vector x of the drug molecule i Decompose into h head features H of equal size ik , and at the same time, the feature vector x is segmented i Mapping to multiple subspaces:

[0058] Split(x i )=(H i1 ,H i2 ,…,H ih )k=1,2,…,h

[0059] Each head feature H ik Through an Adapter ik Processing, Adapter ikIt includes: down-projection layer, nonlinear activation function and up-projection layer; the down-projection layer maps the head feature to a lower dimension to obtain the low-dimensional feature x down , and then the nonlinear expression ability of the model is increased through the nonlinear activation function, and finally the upper projection layer maps the features back to the original dimension to obtain the original dimension feature x up ;

[0060] Adapter ik The output H′ ik for:

[0061] Adapter(x)=Relu(W down x+b down )W up +b up

[0062] H′ ik =Adapter ik (H ik )k=1,2,3,…,h

[0063] in, represents the downsampling matrix, represents the downsampling bias, represents the upsampling matrix, Represents the upsampling bias; M represents the dimension of the input x, R represents the bottleneck dimension, R<<M;

[0064] Head characteristics H′ after all adapters have processed ik The concatenation operation is combined with the feature vector x of the drug molecule. i Combined through residual connections, the fine-tuned output feature x′ is formed i :

[0065] x′ i =Concat(H′ i1 ,H′ i2 ,…,H′ ih )+x i

[0066] The fine-tuned output feature x′ i Input the trained drug molecular property prediction network to obtain the drug molecular properties.

[0067] Embodiment 2:

[0068] This embodiment takes the Transformer model as an example to provide a molecular property prediction method based on a head feature parallel adapter (HFPA), comprising the following steps:

[0069] Step 1: Freeze the backbone of the pre-trained model. Taking the Transformer model as an example, insert the head feature parallel adapter between the feedforward layer and the normalization layer.

[0070] In this embodiment, the backbone of the pre-trained Transformer model is frozen to keep the general feature representation obtained in the self-supervised learning task. Then, starting from the second layer, head feature parallel adapters are inserted between the feedforward network and the subsequent layer normalization of each Transformer layer. These head feature parallel adapters introduce additional trainable parameters to specifically adjust the feature representation for specific drug discovery tasks while ensuring that the basic knowledge of the model is not lost, thereby enhancing the adaptability and prediction accuracy of the model to new tasks while maintaining the original effective feature extraction capability.

[0071] Among them, the head feature parallel adapter includes: a pre-trained model feature extraction module, a head feature segmentation module, a head feature learning module, and a head feature merging module.

[0072] The pre-trained model feature extraction module is based on the Transformer architecture. It processes sequence data through self-attention mechanism and position encoding, and uses self-supervised learning tasks to train on large-scale unlabeled molecular data to capture molecular features and form effective sequence representations.

[0073] The head feature segmentation module is responsible for segmenting the feature vector output by the Transformer model into multiple subspaces, each of which represents a head feature, providing a basis for parallel processing and feature learning.

[0074] The head feature learning module equips each segmented head feature with an adapter, which consists of a fully connected layer and a nonlinear activation function, focusing on learning and optimizing the molecular feature representation in each subspace.

[0075] The head feature merging module integrates multiple head features processed by the adapter into a complete feature vector through splicing or other merging strategies, ensuring that the model can utilize comprehensive molecular information when making property predictions.

[0076] Based on the above structure, the head parallel adapter calculation process includes:

[0077] First, receive the molecular feature vector x from the i-th layer encoder of the pre-trained Transformer model i , which contains rich molecular representation information. In order to adapt to specific drug discovery tasks, this example uses x i Decomposed into h head features H of equal size ik ,like Figure 3As shown in Figure 1, each head feature focuses on learning different physicochemical properties of drug molecules, such as molecular shape and hydrogen bonding. This feature space decomposition process does not introduce additional parameters, ensuring the parameter efficiency of the model, while mapping the feature vector to multiple subspaces through the segmentation operation:

[0078] Split(x i )=(H i1 ,H i2 ,…,H ih )k=1,2,…,h

[0079] Next, each head feature H ik Through an Adapter ik For processing, the adapter consists of a down-projection layer, a non-linear activation function (such as ReLU), and an up-projection layer.

[0080] The down-projection layer first projects the head features to a lower dimension x down , and then increase the nonlinear expression ability of the model through the nonlinear activation function, and finally the projection layer maps the features back to the original dimension x up The output of the adapter is H′ ik =x up Provides finer adjustments for each head feature to better suit the needs of subsequent tasks:

[0081] Adapter(x)=ReLU(W down x+b down )W up +b up

[0082] H′ ik =Adapter ik (H ik )k=1,2,3,…,h

[0083] Head characteristics H′ after all adapters have processed ik Merge through concatenation operation and combine with the original feature x i Combined through residual connections, the fine-tuned output feature x′ is formed i :

[0084] x′ i =Concat(H′ i1 ,H′ i2 ,…,H′ ih )+x i

[0085] This feature merging process not only retains the model’s prior knowledge, but also enhances the model’s adaptability to specific tasks by introducing adapters.

[0086] In order to further promote the interaction between head features and make full use of the potential connections between drug molecular features, this embodiment also introduces a head feature parallel cross adapter (HFPCA), which is a supplementary solution of HFPA. In practical applications, both HFPA and HFPCA can be selected. Compared with HFPA, HFPCA has an additional Figure 3 The “cross” module shown in the middle. In this “cross” module, all low-dimensional adapter output features are first concatenated for joint learning. Then through a fully connected layer is processed to learn the interaction between head features. Finally, the segmentation operation Match the processed cross features to the number and dimensions of the input features and pass them into the Adapter ik The projection layer maps back to the original dimension x up , the subsequent operations are consistent with HFPA, and finally the fine-tuned feature x′ is obtained i .

[0087] Through this coherent computational process, HFPA and HFPCA enable the pre-trained model to adjust its feature representation more accurately and adapt to specific drug discovery tasks by inputting it into the drug molecular property prediction network of downstream tasks, while maintaining the interpretability and predictive performance of the model. This computational framework not only improves the performance of the model in complex drug discovery tasks, but also provides an efficient and flexible solution for the field of drug discovery by optimizing feature representation and enhancing feature interaction.

[0088] Step 2: Obtain molecular data for a specific task, encode the molecular sequence, obtain its vector representation, and divide it into training and test sets;

[0089] This embodiment collects relevant molecular data for a specific drug discovery task, converts the structural information of each molecule into a SMILES string or other encoding form, and then converts it into a numerical vector representation through chemical informatics methods. Finally, these vectors are divided into a training set for model training and a test set for performance evaluation, ensuring that the data set can both represent the task characteristics and meet the needs of model training and verification.

[0090] Step 3: Use the pre-trained model to extract deep features from the molecular sequence as the initial hidden state of the head feature parallel adapter, and focus on exploring the atomic and chemical bond knowledge of different feature subspaces through different small adapters to maintain fewer trainable parameters and enhance the interpretability of the model. By training the head feature parallel adapter and the molecular property prediction network, a trained specific molecular property prediction model is obtained.

[0091] Step 4: Use the trained molecular property prediction model to predict the molecules, complete the efficient automatic calculation of molecular properties, and allow comprehensive explanatory analysis to provide valuable insights into the field of chemistry.

[0092] Embodiment three:

[0093] In this embodiment, a molecular property prediction network based on the head parallel adapter (HFPA) for model fine-tuning of the present invention is compared with the existing prediction model, and the experimental results are compared by means of scientific demonstration to verify the technical effect of the method of the present invention.

[0094] This example adopts the head feature parallel adapter (HFPA) fine-tuning strategy, configures two head features to fine-tune the pre-trained model for a drug discovery task specific to the blood-brain barrier (BBBP) dataset, and compares it with the traditional global fine-tuning (FT) method.

[0095] See also Figure 4 , the experimental results show that in the HFPA model with two head features, the cosine similarity between the head features is significantly lower than that of the global fine-tuning model, which indicates the effectiveness of HFPA in reducing feature redundancy. Specifically, low cosine similarity means that each head feature of the HFPA model can capture more independent and complementary information, thereby enhancing the model's ability to explain different molecular properties. This ability to reduce redundancy and enhance feature independence is crucial to revealing the complexity and diversity of molecules, and provides a more accurate and reliable model for predicting molecular properties in the drug discovery process.

[0096] See also Figure 5 The primary head features in this example model focus on electronegative atoms (F, Cl, Br, and I) and functional groups (amino, hydroxyl, and carboxyl), all of which are known to affect lipophilicity (a key factor in molecule permeability). Meanwhile, the secondary head features focus on molecular stereochemistry, with an emphasis on ring structure and branching, which are critical for crossing the blood-brain barrier.

[0097] In addition, the experiment of this embodiment deeply studies the influence of the hyperparameter selection on the performance of the prediction model of the present invention, including the dimension r of each head feature and the number h of head features.

[0098] Fig. 6A and Figure 6BThe substantial changes in model performance due to hyperparameter selection are described. To evaluate the significance of hyperparameters, the fANOVA algorithm is used in this embodiment. The impact of hyperparameters is different on different datasets. For biological activity prediction datasets such as BBBP and BACE, the number of head features h is crucial; on the contrary, for the FreeSolv regression dataset, the size of the hidden layer of the head feature r is shown to be more influential. The performance of the prediction model of the present invention remains robust in various parameter combinations, demonstrating the effectiveness of the model fine-tuning method based on the head parallel adapter (HFPA) of the present invention.

[0099] The choice of hyperparameters can be made based on the number of predicted targets. For single-label classification tasks such as the BBBP dataset, using two head features is consistent with the cross entropy loss function; for multi-label tasks such as Tox21, choosing a number of head features that is a multiple of the number of labels often produces strong results. Figure 6C It can be seen that, except for the six head features, the AUROC scores of most parameter combinations are around 0.830, and when three head features are used with each hidden layer size of 64, the AUROC peak of the performance is 0.832.

[0100] It is important to define the ideal hidden layer dimension for head features. Larger dimensions often yield favorable results where overfitting does not dominate - experimenting with dimensions like 64, 32, 16, and 6 often yields satisfactory performance. Especially in regression scenarios, considering larger dimensions and exploring HFPCA structures may offer favorable prospects.

[0101] When evaluating the quality of a feature space, alignment metrics are used to measure the proximity of similar features in neighboring embeddings. Figure 7 We describe the approach taken by HFPA to generate representations of common molecular scaffolds on different datasets and compare them to representations produced by pre-trained models and adapter fine-tuning.

[0102] This example uses t-distributed stochastic neighbor embedding (t-SNE) for visual evaluation, which focuses on the molecular scaffold, that is, the basic structure inside the molecule. Changes in the scaffold usually correspond to significant differences in chemical properties.

[0103] For evaluation, 6 common scaffolds were selected from each dataset (HIV, TOX21, lipophilicity) and distinguished by different colors during visualization. The pre-trained model, although not fine-tuned for a specific task, demonstrated an excellent ability to capture effective molecular representations, which highlights its intrinsic ability to learn valuable molecular features; in contrast, the model fine-tuned with the Adapter module, while capable of downstream tasks, showed signs of information loss, especially in the Tox21 and HIV datasets, where the results became apparent through the Davies-Bouldin (DB) index, indicating that cluster separation was reduced compared to the pre-trained model without downstream fine-tuning. The prediction network based on the head feature parallel adapter (HFPA) of this embodiment consistently showed the lowest DB index among all models, indicating that it has the best clustering performance.

[0104] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.

[0105] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A molecular property prediction and fine-tuning method, characterized in that: The method comprises: introducing a head feature parallel adapter into a pre-trained model for drug molecular property prediction, freezing the main part of the pre-trained model, and obtaining a trained specific molecular property prediction model by training the head feature parallel adapter and a drug molecular property prediction network; The calculation process of the head feature parallel adapter includes: First, the feature extraction part of the pre-trained model is used to extract the feature vector x of the drug molecule. i , the characteristic vector x of the drug molecule i Decompose into h head features H of equal size ik , and at the same time, the feature vector x is segmented i Mapping to multiple subspaces: Split(x i )=(H i1 ,H i2 ,…,H ih )k=1,2,…,h Each head feature H ik Through an Adapter ik For processing, the adapter ik It includes: a down-projection layer, a nonlinear activation function and an up-projection layer; the down-projection layer maps the head feature to a lower dimension to obtain a low-dimensional feature x down , and then the nonlinear expression ability of the model is increased by the nonlinear activation function. Finally, the upper projection layer maps the feature back to the original dimension to obtain the original dimension feature x up ; The Adapter ik The output H′ ik for: Adapter(x)=Relu(W down x+b dowm )IN up +b up H′ ik =Adapter ik (H ik )k=1,2,3,…,h in, represents the downsampling matrix, represents the downsampling bias, represents the upsampling matrix, Represents the upsampling bias; M represents the dimension of the input x, R represents the bottleneck dimension, R<<M; Head characteristics H′ after all adapters have processed ik The drug molecule feature vector x is merged by the concatenation operation and i Combined through residual connections, the fine-tuned output feature x′ is formed i : x′ i =Concat(H′ i1 ,H′ i2 ,…,H′ ih )+x i The fine-tuned output feature x′ i Input the trained drug molecular property prediction network to obtain the drug molecular properties.

2. The molecular property prediction and fine-tuning method according to claim 1, characterized in that: The method also includes: introducing a head feature parallel cross adapter HFPCA, all low-dimensional adapter output features are first concatenated for joint learning Then through a fully connected layer is processed to learn the interaction between head features; finally, the segmentation operation Match the processed cross features to the number and dimensions of the input features.

3. The molecular property prediction and fine-tuning method according to claim 1, characterized in that: The method further includes: converting the structural information of the drug molecule into a SMILES string or other encoding forms, and converting it into a numerical vector representation through a cheminformatics method.

4. The molecular property prediction and fine-tuning method according to claim 1, characterized in that: The method adopts a Transformer model and, starting from the second layer of the Transformer model, inserts the head feature parallel adapter between a feed-forward network of each Transformer layer and a subsequent layer normalization.

5. A molecular property prediction and fine-tuning system, characterized in that: The system introduces a head feature parallel adapter into a pre-trained model for drug molecular property prediction, freezes the main part of the pre-trained model, and obtains a trained specific molecular property prediction model by training the head feature parallel adapter and a drug molecular property prediction network; The head feature parallel adapter includes: a head feature segmentation module, a head feature learning module, and a head feature merging module; The head feature segmentation module is used to extract the feature vector x of the drug molecule from the pre-training model. i Decompose into h head features H of equal size ik , and at the same time, the feature vector x is segmented i Mapping to multiple subspaces: Split(x i )=(H i1 ,H i2 ,…,H ih )k=1,2,…,h The head feature learning module performs a ik Through an Adapter ik For processing, the adapter ik It includes: a down-projection layer, a nonlinear activation function and an up-projection layer; the down-projection layer maps the head feature to a lower dimension to obtain a low-dimensional feature x down , and then the nonlinear expression ability of the model is increased by the nonlinear activation function. Finally, the upper projection layer maps the feature back to the original dimension to obtain the original dimension feature x up ; The Adapter ik The output H′ ik for: Adapter(x)=Relu(W down x+b down )IN up +b up H′ ik =Adapter ik (H ik )k=1,2,3,…,h in, represents the downsampling matrix, represents the downsampling bias, represents the upsampling matrix, Represents the upsampling bias; M represents the dimension of the input x, R represents the bottleneck dimension, R<<M; The head feature merging module combines all the head features H′ processed by the adapter ik The drug molecule feature vector x is merged by the concatenation operation and i Combined through residual connections, the fine-tuned output feature x′ is formed i : x′ i =Concat(H′ i1 ,H′ i2 ,…,H′ ih )+x i The fine-tuned output feature x′ i Input the trained drug molecular property prediction network to obtain the drug molecular properties.

6. The molecular property prediction and fine-tuning system according to claim 5, characterized in that: The system also includes a head feature parallel cross adapter HFPCA, where all low-dimensional adapter output features are first concatenated for joint learning Then through a fully connected layer is processed to learn the interaction between head features; finally, the segmentation operation Match the processed cross features to the number and dimensions of the input features.

7. The molecular property prediction and fine-tuning system according to claim 5, characterized in that: The system converts the structural information of drug molecules into SMILES strings or other encoding forms, and converts them into numerical vector representations through cheminformatics methods.

8. The molecular property prediction and fine-tuning system according to claim 5, characterized in that: The system adopts a Transformer model, and starting from the second layer of the Transformer model, the head feature parallel adapter is inserted between the feedforward network of each Transformer layer and the subsequent layer normalization.

9. An electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the molecular property prediction and fine-tuning method according to any one of claims 1 to 4 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the molecular property prediction and fine-tuning method according to any one of claims 1 to 4 is implemented.

Images