A molecular property prediction fine-tuning method based on a drug pre-training model

Abstract

The application discloses a kind of molecular property prediction fine-tuning methods based on drug pre-training model, belong to drug discovery technical field.The method introduces head feature parallel adapter in pre-training model, after freezing the main part of pre-training model, through training head feature parallel adapter and drug molecular property prediction network, the specific molecular property prediction model trained is obtained;Head feature parallel adapter realizes in-depth learning to the complex chemical space inside molecule by splitting, adapting and merging molecular feature vector, adapter is composed of fully connected layer and nonlinear activation function, focuses on optimizing molecular feature representation, while maintaining the efficiency of model parameters, not only improves the performance of model, but also speeds up the process of drug discovery, reduces experimental cost, and because of its wide applicability, these adapters can be generalized to materials science and environmental science and many other fields, provide an efficient and flexible tool for the research of related fields.

Description

Technical Field

[0001] This invention relates to a method for predicting and fine-tuning molecular properties based on a drug pre-training model, belonging to the field of drug discovery technology. Background Technology

[0002] Drug discovery is a complex and resource-intensive process. Traditional methods rely on experiments and clinical trials, which are not only time-consuming but also costly. The development of artificial intelligence, particularly the application of deep learning in bioinformatics, has brought new possibilities to drug discovery. Pre-trained models, such as Graph Neural Networks, Graph Transformer Networks, and BERT, have shown great potential in drug discovery, capable of acquiring comprehensive representations of molecular structures through generalization learning on unlabeled datasets. These models can predict a wide range of molecular properties, from quantum chemistry to physiology, and are used as feature encoders.

[0003] However, these large networks face challenges in fine-tuning downstream tasks because they require storing and deploying separate copies of model parameters for each task, which is not only costly but often impractical. To overcome these limitations, researchers have proposed Parameter-Efficient Fine-Tuning (PEFT) methods. These methods improve efficiency by freezing the background network and potentially introducing additional trainable parameters, specifying a set of parameters for training, or redefining the adaptation process. While these PEFT methods have performed well in natural language processing and computer vision tasks, their application in drug discovery has been rarely studied.

[0004] Furthermore, existing PEFT methods face challenges in drug discovery applications because they employ a one-size-fits-all strategy, integrating uniform parameters into all downstream knowledge. While this uniformity simplifies the fine-tuning process, it neglects the multifaceted knowledge inherent in pre-trained models and the multi-factor requirements of specific downstream tasks, resulting in weaker models in explicitly learning the complex chemical spaces within molecules. In drug discovery, tasks requiring comprehensive knowledge are ubiquitous, and strong interpretability is crucial in this field. Currently, few studies have focused on enhancing the interpretability and predictive power of models by learning various aspects of prior knowledge and downstream information during the fine-tuning process, which remains a challenging task. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a molecular property prediction fine-tuning method based on a drug pre-trained model. This method is particularly suitable for the field of drug discovery. It achieves effective fine-tuning of the pre-trained model by designing a highly interpretable and high-performance fine-tuning paradigm, namely the Head Feature Parallel Adapter (HFPA). The technical solution is as follows:

[0006] The molecular property prediction fine-tuning method of the present invention includes: introducing a head feature parallel adapter into a pre-trained model for drug molecular property prediction; freezing the backbone of the pre-trained model; and obtaining a trained specific molecular property prediction model by training the head feature parallel adapter and the drug molecular property prediction network.

[0007] The calculation process of the head feature parallel adapter includes:

[0008] First, the feature vector x of the drug molecule is extracted using the feature extraction part of the pre-trained model. i The feature vector x of the drug molecule i Decompose into h head features of equal size H ik Simultaneously, the feature vector x is segmented through a segmentation operation. i Mapped to multiple subspaces:

[0009] Split(x i )=(Hi1,Hi2,…,Hih)k=1,2,…,h

[0010] Each header feature H ik Through an adapter ik The adapter is processed. ik It includes: a lower projection layer, a non-linear activation function, and an upper projection layer; the lower projection layer maps the head features to a lower dimension, resulting in low-dimensional features x. down Then, the nonlinear activation function is used to enhance the nonlinear expressive power of the model. Finally, the upper projection layer maps the features back to the original dimension to obtain the original dimension features x. up ;

[0011] The adapter ik The output H′ ik for:

[0012] Adapter(x) = Relu(W) down x+b down W up +b up

[0013] H′ik =Adapter ik (H ik k = 1, 2, 3, ..., h

[0014] in, Represents the downsampling matrix. Indicates the downsampling bias. Represents the upsampling matrix. R represents the upsampling bias; M represents the dimension of the input x, R represents the bottleneck dimension, and R << M;

[0015] Header feature H′ after all adapter processing ik The components are merged through a splicing operation and then combined with the feature vector x of the drug molecule. i By combining residual connections, the fine-tuned output feature x′ is formed. i :

[0016] x′ i =Concat(H′) i1 ,H′ i2 ,…,H′ ih )+x i

[0017] The fine-tuned output feature x′ i Input the trained drug molecule property prediction network to obtain the drug molecule properties.

[0018] Optionally, the method further includes: introducing a head feature parallel cross-adaptor HFPCA, where all low-dimensional adapter output features are first concatenated for joint learning. Then through a fully connected layer Processing is performed to learn the interactions between head features; finally, through segmentation operations... The number and dimension of the input features are matched with the processed cross features.

[0019] Optionally, the method further includes: converting the structural information of the drug molecule into a SMILES string or other encoded form, and converting it into a numerical vector representation using cheminformatics methods.

[0020] Optionally, the method employs a Transformer model, and starting from the second layer of the Transformer model, inserts the head feature parallel adapter between the feedforward network of each Transformer layer and the subsequent layer normalization.

[0021] This invention provides a molecular property prediction fine-tuning system. The system introduces a head feature parallel adapter into a pre-trained model for drug molecular property prediction. After freezing the backbone of the pre-trained model, a trained specific molecular property prediction model is obtained by training the head feature parallel adapter and the drug molecular property prediction network.

[0022] The head feature parallel adapter includes: a head feature segmentation module, a head feature learning module, and a head feature merging module;

[0023] The head feature segmentation module is used to segment the feature vector x of the drug molecule extracted by the pre-trained model. i Decompose into h head features of equal size H ik Simultaneously, the feature vector x is segmented through a segmentation operation. i Mapped to multiple subspaces:

[0024] Split(x i )=(H i1 H i2 ,…,H ih k = 1, 2, ..., h

[0025] The head feature learning module processes each head feature H. ik Through an adapter ik The adapter is processed. ik It includes: a lower projection layer, a non-linear activation function, and an upper projection layer; the lower projection layer maps the head features to a lower dimension, resulting in low-dimensional features x. down Then, the nonlinear activation function is used to enhance the nonlinear expressive power of the model. Finally, the upper projection layer maps the features back to the original dimension to obtain the original dimension features x. up ;

[0026] The adapter ik The output H′ ik for:

[0027] Adapter(x) = Relu(W) down x+b down W up +b up

[0028] H′ ik =Adapter ik (H ik k = 1, 2, 3, ..., h

[0029] in, Represents the downsampling matrix. Indicates the downsampling bias. Represents the upsampling matrix. R represents the upsampling bias; M represents the dimension of the input x, R represents the bottleneck dimension, and R << M;

[0030] The header feature merging module merges all the header features H′ processed by the adapters. ik The components are merged through a splicing operation and then combined with the feature vector x of the drug molecule. i By combining residual connections, the fine-tuned output feature x′ is formed. i :

[0031] x′ i =Concat(H′) i1 ,H′ i2 ,…,H′ ih )+x i

[0032] The fine-tuned output feature x′ i Input the trained drug molecule property prediction network to obtain the drug molecule properties.

[0033] Optionally, the system also includes a head feature parallel cross-adaptor (HFPCA), where all low-dimensional adapter output features are first concatenated for joint learning. Then through a fully connected layer Processing is performed to learn the interactions between head features; finally, through segmentation operations... The number and dimension of the input features are matched with the processed cross features.

[0034] Optionally, the system converts the structural information of drug molecules into SMILES strings or other encoded forms, and then transforms them into numerical vector representations using cheminformatics methods.

[0035] Optionally, the system employs a Transformer model, and starting from the second layer of the Transformer model, inserts the head feature parallel adapter between the feedforward network of each Transformer layer and the subsequent layer normalization.

[0036] This invention provides an electronic device, including a memory and a processor;

[0037] The memory is used to store computer programs;

[0038] The processor is configured to implement the molecular property prediction fine-tuning method as described in any of the preceding claims when executing the computer program.

[0039] The present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the molecular property prediction fine-tuning method as described in any of the preceding claims.

[0040] The beneficial effects of this invention are:

[0041] The molecular property prediction fine-tuning method provided by this invention introduces a head feature parallel adapter (HFPA) and its extended head feature parallel cross adapter (HFPCA) into the pre-trained model for drug molecular property prediction, and provides a fine-tuning mechanism that significantly improves the efficiency and adaptability of drug discovery tasks while reducing the demand for computing resources.

[0042] Furthermore, the HFPA-based molecular property prediction fine-tuning method of this invention effectively improves the accuracy of molecular property prediction by optimizing feature representation and enhancing model interpretability, enabling the model to gain a deeper understanding of the physicochemical properties of drug molecules. Experimental results show that by reducing model parameters and promoting interactions between features, HFPA and HFPCA not only improve model performance but also accelerate drug discovery, reduce experimental costs, and due to their wide applicability, these adapters can be extended to multiple fields such as materials science and environmental science, providing an efficient and flexible tool for research in related fields. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram of the basic process of a molecular property prediction and fine-tuning method provided in one embodiment of the present invention.

[0045] Figure 2 This is a schematic diagram illustrating the location of the head feature parallel adapter embedded in the Transformer model according to an embodiment of the present invention.

[0046] Figure 3 This is a schematic diagram of a parallel adapter structure for head features provided in one embodiment of the present invention.

[0047] Figure 4 This is a schematic diagram illustrating the explainability of a molecular property prediction fine-tuning method provided in one embodiment of the present invention.

[0048] Figure 5This is a schematic diagram illustrating the interpretability analysis of the head feature of a molecular property prediction fine-tuning method provided in one embodiment of the present invention.

[0049] Figure 6A This is a diagram illustrating the substantial change in model performance caused by hyperparameter selection in one embodiment of the present invention.

[0050] Figure 6B This is a graph illustrating the importance of hyperparameter selection to AUROC in one embodiment of the present invention.

[0051] Figure 6C This is a diagram showing the relationship between hyperparameter selection and AUROC in one embodiment of the present invention.

[0052] Figure 7 This is a schematic diagram of the chemical feature space quality of a molecular property prediction fine-tuning method in one embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0054] Example 1:

[0055] This embodiment provides a method for predicting drug molecular properties, including: introducing a head feature parallel adapter into a pre-trained model for predicting drug molecular properties, freezing the backbone of the pre-trained model, and obtaining a trained specific molecular property prediction model by training the head feature parallel adapter and the drug molecular property prediction network.

[0056] The computation process of the head feature parallel adapter includes:

[0057] First, the feature vector x of the drug molecule is extracted using the feature extraction part of the pre-trained model. i The feature vector x of the drug molecule i Decompose into h head features of equal size H ik Simultaneously, the feature vector x is segmented through a segmentation operation. i Mapped to multiple subspaces:

[0058] Split(x i )=(H i1 H i2 ,…,H ih k = 1, 2, ..., h

[0059] Each header feature H ik Through an adapter ik Processing is required; Adapter ikIt includes: a downprojection layer, a non-linear activation function, and an upprojection layer; the downprojection layer maps the head features to a lower dimension, resulting in low-dimensional features x. down Then, a nonlinear activation function is used to increase the nonlinear expressive power of the model. Finally, a projection layer is applied to map the features back to the original dimension, obtaining the original dimension features x. up ;

[0060] Adapter ik The output H′ ik for:

[0061] Adapter(x) = Relu(W) down x+b down W up +b up

[0062] H′ ik =Adapter ik (H ik k = 1, 2, 3, ..., h

[0063] in, Represents the downsampling matrix. Indicates the downsampling bias. Represents the upsampling matrix. R represents the upsampling bias; M represents the dimension of the input x, R represents the bottleneck dimension, and R << M;

[0064] Header feature H′ after all adapter processing ik The components are merged through a splicing operation and then combined with the feature vector x of the drug molecule. i By combining residual connections, the fine-tuned output feature x′ is formed. i :

[0065] x′ i =Concat(H′) i1 ,H′ i2 ,…,H′ ih )+x i

[0066] The fine-tuned output feature x′ i Input the trained drug molecule property prediction network to obtain the drug molecule properties.

[0067] Example 2:

[0068] This embodiment uses the Transformer model as an example to provide a molecular property prediction method based on Head Feature Parallel Adapter (HFPA), including the following steps:

[0069] Step 1: Freeze the backbone of the pre-trained model. Taking the Transformer model as an example, insert a head feature parallel adapter between the feedforward layer and the normalization layer.

[0070] This embodiment freezes the backbone of the pre-trained Transformer model to maintain its general feature representations obtained in self-supervised learning tasks. Then, starting from the second layer, head feature parallel adapters are inserted between the feedforward network of each Transformer layer and the subsequent layer normalization. These head feature parallel adapters adjust the feature representations specifically for drug discovery tasks by introducing additional trainable parameters, while ensuring that the model's fundamental knowledge is not lost. This enhances the model's adaptability to new tasks and prediction accuracy while maintaining its original effective feature extraction capabilities.

[0071] The head feature parallel adapter includes: a pre-trained model feature extraction module, a head feature segmentation module, a head feature learning module, and a head feature merging module.

[0072] The pre-trained model feature extraction module is based on the Transformer architecture. It processes sequence data through self-attention mechanism and position encoding, and is trained on large-scale unlabeled molecular data using a self-supervised learning task to capture molecular features and form effective sequence representations.

[0073] The head feature segmentation module is responsible for dividing the feature vector output by the Transformer model into multiple subspaces, each subspace representing a head feature, providing a foundation for parallel processing and feature learning.

[0074] The head feature learning module equips each segmented head feature with an adapter, which consists of a fully connected layer and a non-linear activation function, focusing on learning and optimizing the molecular feature representation within each subspace.

[0075] The head feature merging module integrates multiple head features processed by the adapter into a complete feature vector through splicing or other merging strategies, ensuring that the model can utilize comprehensive molecular information when making property predictions.

[0076] Based on the above structure, the head parallel adapter computation process includes:

[0077] First, the molecular feature vector x is received from the i-th layer encoder of the pre-trained Transformer model. i This vector contains rich molecular representation information. To adapt to specific drug discovery tasks, this embodiment will use x... i Decompose into h equal-sized head features H ik ,like Figure 3As shown, each head feature focuses on learning different physicochemical properties of drug molecules, such as molecular shape and hydrogen bonds. This feature space decomposition process does not introduce additional parameters, ensuring the parameter efficiency of the model, while mapping the feature vector to multiple subspaces through a segmentation operation:

[0078] Split(x i )=(H i1 H i2 ,…,H ih k = 1, 2, ..., h

[0079] Next, each head feature H ik Through an adapter ik The adapter is processed by consisting of a lower projection layer, a nonlinear activation function (such as ReLU), and an upper projection layer.

[0080] The lower projection layer first maps the head features to a lower dimension x. down Then, a non-linear activation function is used to enhance the model's non-linear expressive power, and finally, a projection layer is applied to map the features back to the original dimension x. up The adapter's output H′ ik =x up More fine-tuning was provided for each head feature to better adapt it to the needs of subsequent tasks:

[0081] Adapter(x) = ReLU(W) down x+b down W up +b up

[0082] H′ ik =Adapter ik (H ik k = 1, 2, 3, ..., h

[0083] Header feature H′ after all adapter processing ik The features are merged through a concatenation operation and then combined with the original feature x. i By combining residual connections, the fine-tuned output feature x′ is formed. i :

[0084] x′ i =Concat(H′) i1 ,H′ i2 ,…,H′ ih )+x i

[0085] This feature merging process not only preserves the model's prior knowledge, but also enhances the model's adaptability to specific tasks by introducing an adapter.

[0086] To further facilitate interactions between head features and fully utilize the potential connections between drug molecule features, this embodiment also introduces a head feature parallel cross-adaptor (HFPCA), a supplement to HFPA. In practical applications, either HFPA or HFPCA can be chosen. Compared to HFPA, HFPCA has an additional feature such as... Figure 3 The "cross" module shown in the middle. In this "cross" module, all the low-dimensional adapter output features are first concatenated for joint learning. Then through a fully connected layer Processing is performed to learn the interactions between head features. Finally, through segmentation operations... The processed cross-features are matched against the number and dimensions of the input features and then passed to the Adapter. ik Mapping back to the original dimension x in the upper projection layer up The subsequent operations are the same as HFPA, and finally the fine-tuned feature x′ is obtained. i .

[0087] Through this coherent computational process, HFPA and HHFPCA enable pre-trained models to more accurately adjust their feature representations and adapt them to specific drug discovery tasks by inputting them into downstream drug molecule property prediction networks, while maintaining the model's interpretability and predictive performance. This computational framework not only improves the model's performance in complex drug discovery tasks but also provides an efficient and flexible solution for the field of drug discovery by optimizing feature representations and enhancing feature interactions.

[0088] Step 2: Obtain molecular data for a specific task, encode the molecular sequences to obtain their vector representations, and divide them into training and testing sets;

[0089] This embodiment collects relevant molecular data for a specific drug discovery task, converts the structural information of each molecule into SMILES strings or other encoded forms, and then uses cheminformatics methods to convert them into numerical vector representations. Finally, these vectors are divided into a training set for model training and a test set for performance evaluation, ensuring that the dataset can both represent the characteristics of the task and meet the needs of model training and validation.

[0090] Step 3: Extract deep features from the molecular sequence using a pre-trained model, which serve as the initial hidden state of the head feature parallel adapter. Different small adapters are used to focus on exploring atomic and chemical bond knowledge in different feature subspaces to maintain fewer trainable parameters and enhance the interpretability of the model. The trained specific molecular property prediction model is obtained by training the head feature parallel adapter and the molecular property prediction network.

[0091] Step 4: Use the trained molecular property prediction model to predict molecular properties, complete the efficient and automatic calculation of molecular properties, and allow for comprehensive interpretive analysis, providing valuable insights for the field of chemistry.

[0092] Example 3:

[0093] This embodiment compares the molecular performance prediction network based on head parallel adapter (HFPA) for model fine-tuning of the present invention with existing prediction models, and compares the experimental results with scientific verification methods to verify the technical effect of the method of the present invention.

[0094] This embodiment employs a Head Feature Parallel Adapter (HFPA) fine-tuning strategy. For the drug discovery task specific to the Blood-Brain Barrier (BBBP) dataset, two head features are configured to fine-tune the pre-trained model, and a comparison is made with the traditional Global Fine-tuning (FT) method.

[0095] See Figure 4 Experimental results show that in the HFPA model with two head features, the cosine similarity between head features is significantly lower than that in the globally fine-tuned model. This phenomenon indicates the effectiveness of HFPA in reducing feature redundancy. Specifically, low cosine similarity means that each head feature of the HFPA model can capture more independent and complementary information, thereby enhancing the model's ability to interpret different molecular properties. This ability to reduce redundancy and enhance feature independence is crucial for revealing the complexity and diversity of molecules, providing a more accurate and reliable model for predicting molecular properties in the drug discovery process.

[0096] See Figure 5 In this embodiment, the primary head features are concentrated on electronegative atoms (F, Cl, Br, and I) and functional groups (amino, hydroxyl, and carboxyl groups), all of which are known to affect lipophilicity (a key factor in molecular permeability). Meanwhile, secondary head features focus on molecular stereochemistry, particularly ring structures and branching, which are crucial for crossing the blood-brain barrier.

[0097] Furthermore, the experiments in this embodiment thoroughly investigated the impact of hyperparameter selection on the performance of the prediction model of this invention, including the dimension r of each head feature and the number of head features h.

[0098] Figure 6A and Figure 6BThis paper describes the substantial changes in model performance caused by hyperparameter selection. To evaluate the significance of hyperparameters, the fANOVA algorithm is used in this embodiment. The impact of hyperparameters varies across different datasets. For bioactivity prediction datasets such as BBBP and BACE, the number of head features, h, is crucial; conversely, for the FreeSolv regression dataset, the size of the hidden layer of the head features r has proven to be more influential. The performance of the prediction model in this invention remains robust across various parameter combinations, demonstrating the effectiveness of the model fine-tuning method based on the Head Parallel Adapter (HFPA).

[0099] Hyperparameters can be chosen based on the number of targets to be predicted. For single-label classification tasks like the BBBP dataset, using two head features is consistent with the cross-entropy loss function; for multi-label tasks like Tox21, choosing a head feature count that is a multiple of the number of labels often yields powerful results. Figure 6C As can be seen, apart from the six head features, the AUROC scores of most parameter combinations are around 0.830. When using three head features and each hidden layer size of 64, the peak AUROC score is 0.832.

[0100] Defining the ideal hidden layer dimension for head features is important. Larger dimensions often yield favorable results when overfitting is not dominant. Experimenting with dimensions such as 64, 32, 16, and 6 often produces satisfactory performance, especially in regression scenarios. Considering larger dimensions and exploring HFPCA structures may offer favorable prospects.

[0101] When evaluating the quality of a feature space, alignment metrics are used to measure how close similar features are in adjacent embeddings. Figure 7 This paper describes the methods used by HFPA to generate common molecular scaffold representations on different datasets and compares them with representations generated by pre-trained models and adapter fine-tuning.

[0102] This embodiment utilizes t-distributed random neighbor embedding (t-SNE) for visual evaluation. This method mainly studies molecular scaffolds, i.e., the basic structure inside molecules. Changes in the scaffold usually correspond to significant differences in chemical properties.

[0103] For evaluation, six common scaffolds were selected from each dataset (HIV, TOX21, lipophilic) and distinguished by different colors during visualization. The pre-trained model, while not fine-tuned for specific tasks, demonstrated a superior ability to capture effective molecular representations, highlighting its inherent capacity to learn valuable molecular features. In contrast, models fine-tuned using the Adapter module, while capable of handling downstream tasks, showed signs of information loss, particularly in the Tox21 and HIV datasets. This became apparent through the Davies-Bouldin (DB) index, indicating reduced cluster separation compared to the pre-trained model without downstream fine-tuning. The prediction network based on the Head Feature Parallel Adapter (HFPA) in this embodiment consistently showed the lowest DB index among all models, indicating its superior clustering performance.

[0104] Some steps in the embodiments of the present invention can be implemented using software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk.

[0105] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A molecular property prediction fine-tuning method, characterized in that, The method comprises introducing a head feature parallel adapter into a pre-trained model for drug molecule property prediction, and after freezing the backbone part of the pre-trained model, obtaining a trained specific molecule property prediction model by training the head feature parallel adapter and a drug molecule property prediction network. The calculation process of the head feature parallel adapter comprises: First, the feature vector x of the drug molecule is extracted using the feature extraction part of the pre-trained model. i The feature vector x of the drug molecule i Decompose into h head features of equal size H ik Simultaneously, the feature vector x is segmented through a segmentation operation. i Mapped to multiple subspaces: Split(x i ) = (H i1 ,H i2 ,…,H ih ) k = 1,2,…,h Each head feature H ik Through an adapter Adapter ik Processing, the adapter Adapter ik Including: the lower projection layer, the nonlinear activation function and the upper projection layer; the lower projection layer maps the head feature to a lower dimension, and obtains the low-dimensional feature x down Then through the nonlinear activation function, the nonlinear expression ability of the model is increased, and finally the upper projection layer maps the feature back to the original dimension, and obtains the original dimension feature x up ; The adapter Adapter ik The output H' of the filter Filter ik is: Adapter(x) = Relu(W down x + b dowm )W up + b up H' ik = Adapter ik (H ik )k = 1,2,3,…,h wherein, denotes a down-sampling matrix, denotes a down-sampling bias, denotes an up-sampling matrix, denotes an up-sampling bias; M denotes the dimension of the input x, R denotes the bottleneck dimension, R « M. All adapter processed head features H' ik merged by concatenation operation, and with the feature vector x i combined by residual connection, forming fine-tuned output features x' i : x′ i = Concat(H′ i1 ,H′ i2 ,…,H′ ih )+x i outputting the fine-tuned output features x' i inputting the trained drug molecule property prediction network to obtain a drug molecule property.

2. The molecular property prediction fine-tuning method of claim 1, wherein, The method also includes introducing a head feature parallel cross adapter HFPCA, and all low-dimensional adapter output features are first concatenated for joint learning Then through a fully connected layer Processing is performed to learn the interaction between the head features; finally, through a segmentation operation The processed cross features match the number and dimension of the input features.

3. The molecular property prediction fine-tuning method of claim 1, wherein, The method further comprises converting the structure information of the drug molecule into a SMILES string or other coding form, and converting it into a numerical vector representation through chemoinformatics methods.

4. The molecular property prediction fine-tuning method of claim 1, wherein, The method adopts a Transformer model, and from the second layer of the Transformer model, the head feature parallel adapter is inserted between the feedforward network of each Transformer layer and the subsequent layer normalization.

5. A molecular property prediction fine-tuning system, comprising: The system introduces a head feature parallel adapter into a pre-trained model for drug molecule property prediction, and after freezing the backbone part of the pre-trained model, obtains a trained specific molecule property prediction model by training the head feature parallel adapter and a drug molecule property prediction network. The head feature parallel adapter comprises a head feature segmentation module, a head feature learning module, and a head feature merging module. The head feature segmentation module is configured to segment the feature vector x of the drug molecule extracted by the pre-training model into h head features H of equal size i The head feature segmentation module is configured to segment the feature vector x of the drug molecule extracted by the pre-training model into h head features H of equal size ik The head feature segmentation module is configured to segment the feature vector x of the drug molecule extracted by the pre-training model into h head features H of equal size i The head feature segmentation module is configured to segment the feature vector x of the drug molecule extracted by the pre-training model into h head features H of equal size Split(x i ) = (H i1 , H i2 ,..., H ih ) k = 1, 2,..., h The head feature learning module maps each head feature H ik Through an adapter Adapter ik Processing, the adapter Adapter ik It includes a lower projection layer, a nonlinear activation function and an upper projection layer; the lower projection layer maps the head feature to a lower dimension to obtain a low-dimensional feature x down Then the nonlinear activation function is used to increase the nonlinear expression ability of the model, and finally the upper projection layer maps the feature back to the original dimension to obtain the original dimension feature x up ; The output H' of the adapter Adapter ik is: ik ​ Adapter(x) = Relu(W down x + b down )W up + b up H' ik = Adapter ik (H ik )k = 1,2,3,…,h wherein, denotes a down-sampling matrix, denotes a down-sampling bias, denotes an up-sampling matrix, denotes an up-sampling bias; M denotes the dimension of the input x, R denotes the bottleneck dimension, R « M. The head feature merging module merges all the adapter-processed head features H′ ik by concatenation operation, and the feature vector x of the drug molecule i by residual connection, to form the fine-tuned output feature x′ i : x′ i = Concat(H′ i1 ,H′ i2 ,…,H′ ih )+x i outputting the fine-tuned output features x' i inputting the trained drug molecule property prediction network to obtain a drug molecule property.

6. The molecular property prediction fine-tuning system of claim 5, wherein, The system also includes a head feature parallel cross adapter HFPCA, all low-dimensional adapter output features are first concatenated for joint learning Then through a fully connected layer Processing to learn the interaction between head features; finally, through a segmentation operation The processed cross features match the number and dimension of the input features.

7. The molecular property prediction fine-tuning system of claim 5, wherein, The system converts the structure information of the drug molecule into a SMILES string or other coding form, and converts it into a numerical vector representation through chemoinformatics methods.

8. The molecular property prediction fine-tuning system of claim 5, wherein, The system adopts a Transformer model, and from the second layer of the Transformer model, the head feature parallel adapter is inserted between the feedforward network of each Transformer layer and the subsequent layer normalization.

9. An electronic device, comprising: comprise a memory and a processor; The memory is configured to store a computer program; The processor is configured to implement the molecule property prediction fine-tuning method according to any one of claims 1 to 4 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The storage medium has a computer program stored thereon, and when the computer program is executed by a processor, the molecule property prediction fine-tuning method according to any one of claims 1 to 4 is implemented.

Images