Method and system for molecular pre-training representations fusing smi les sequences and molecular graphs

Abstract

The application provides a molecular pre-training representation method and system fusing SMILES sequences and molecular graphs, and comprises the following steps: obtaining a character sequence in SMILES form and a molecular graph; inputting the character sequence in the SMILES form and the molecular graph into a pre-trained Transformer model for processing from a molecular level and a fragment level respectively, and outputting a molecular representation vector; and taking the molecular representation vector as an input of a downstream task to complete molecular representation. The application solves the problem that the existing molecular representation scheme is not comprehensive and cannot achieve ideal effects.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of compound molecular representation, and in particular to a molecular pre-training representation method and system fusing SMILES sequence and molecular graph. BACKGROUND

[0002] Most molecular representation schemes use single-form data for pre-training, such as using natural language methods for SMILES strings or using graph pre-training methods for molecular graphs. Recently, some schemes have used dual-tower models to use both data forms, such as DMP using SMILES and molecular graphs, MM-Deacon using SMILES and IUPAC names, and CLOOME using molecular graphs and cell micrographs. Such schemes use different branches to encode each data form, and the representations obtained by each branch are trained together through a final loss function.

[0003] Molecular representation schemes using single data forms are relatively limited. Using only SMILES sequences is not conducive to capturing the topological structure of molecules, and using only molecular graphs can cause problems such as excessive smoothing of graph models. Using both SMILES and molecular graphs can complement each other's advantages and obtain more comprehensive molecular representations. However, current schemes using two data forms, such as DMP, use dual-tower architectures to encode different data forms, lack interaction between SMILES and molecular graphs, and ignore the fine-grained correspondence between the two. SUMMARY

[0004] The present application provides a molecular pre-training representation method and system fusing SMILES sequence and molecular graph, to solve the problem of incomplete and unsatisfactory existing molecular representation schemes.

[0005] The present application provides a molecular pre-training representation method fusing SMILES sequence and molecular graph, comprising:

[0006] Obtaining a character sequence in SMILES form and a molecular graph;

[0007] Respectively inputting the character sequence in SMILES form and the molecular graph into a pre-trained Transformer model from the molecular level and the fragment level for processing, and outputting a molecular representation vector;

[0008] Taking the molecular representation vector as input for a downstream task to complete molecular representation.

[0009] According to the molecular pre-training representation method fusing SMILES sequence and molecular graph provided by the present application, the character sequence in SMILES form and the molecular graph are input into a pre-trained Transformer model for processing, specifically comprising:

[0010] At the molecular level, the character sequence in the SMILES form obtains a first embedding vector of each character after passing through a tokenizer and a linear embedding layer;

[0011] The molecular graph obtains a second embedding vector of each node after passing through a graph neural network;

[0012] The first embedding vector and the second embedding vector are jointly input into a Transformer model for processing.

[0013] According to the molecular pre-training representation method for fusing SMILES sequences and molecular graphs provided by the application, the character sequence in the SMILES form and the molecular graph are input into a pre-trained Transformer model for processing, and the method further comprises:

[0014] At the fragment level, the BRICS algorithm is used to divide the molecular graph into fragments to obtain a divided molecular picture fragment;

[0015] The character sequence in the SMILES form is labeled by a labeling rule to obtain a character sequence with labels.

[0016] According to the molecular pre-training representation method for fusing SMILES sequences and molecular graphs provided by the application, the task at the fragment level comprises cross-modal masking and fragment alignment;

[0017] The cross-modal masking is performed at the fragment level and the character level respectively;

[0018] The fragment level masking needs to provide complete information of another modality;

[0019] The character level masking only needs to provide a single modality.

[0020] According to the molecular pre-training representation method for fusing SMILES sequences and molecular graphs provided by the application, the fragment alignment performs contrastive learning on the SMILES fragment representation and the molecular picture fragment representation after average pooling through a multi-head attention mechanism

[0021] The Transformer model is used to capture the fine-grained correspondence between modalities.

[0022] According to the molecular pre-training representation method for fusing SMILES sequences and molecular graphs provided by the application, the molecular representation vector is used as the input of a downstream task to complete molecular representation, and specifically comprises:

[0023] At the molecular level, the character sequence-form molecular graph positive-negative pair is predicted to determine whether they come from the same molecule;

[0024] Through professional knowledge learning to make predictions on molecular fingerprints and functional group information, learning of chemical professional knowledge is strengthened;

[0025] After pre-training, the Transformer model learns cross-modal information at both the fragment and molecule levels to enhance molecular representation.

[0026] The application also provides a molecular pre-training representation system fusing SMILES sequences and molecular graphs, comprising:

[0027] A data acquisition module is configured to acquire character sequences in SMILES form and molecular graphs.

[0028] A processing module is configured to input the character sequences in SMILES form and the molecular graphs into a pre-trained Transformer model for processing from the molecule level and the fragment level respectively, and output molecular representation vectors.

[0029] A molecular representation module is configured to input the molecular representation vectors as inputs of downstream tasks to complete molecular representation.

[0030] The application also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the molecular pre-training representation method fusing SMILES sequences and molecular graphs according to any one of the above methods when executing the program.

[0031] The application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the molecular pre-training representation method fusing SMILES sequences and molecular graphs according to any one of the above methods.

[0032] The application also provides a computer program product comprising a computer program, wherein the computer program is executable on a processor to implement the molecular pre-training representation method fusing SMILES sequences and molecular graphs according to any one of the above methods.

[0033] The application provides a molecular pre-training representation method and system fusing SMILES sequences and molecular graphs, which breaks through the limitations of existing hybrid methods by using a Transformer model and captures fine-grained cross-modal semantics between SMILES sequences and molecular graphs. Corresponding SMILES and molecular pictures are obtained, and these fragments are used as heterogeneous inputs in pre-training tasks together with labels; and a shared transformer is used for deep cross-modal fusion. By introducing two new pre-training tasks at the fragment level, i.e., multi-level cross-modal masking and alignment at the fragment level, fine-grained corresponding relationships between modes are captured, the corresponding relationships between SMILES and molecular graphs at different levels are fully mined, and molecular representation is enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0035] Figure 1 is one of the flowcharts of the molecular pre-training representation method provided by the present application, which fuses SMILES sequence and molecular graph;

[0036] Figure 2 is one of the flowcharts of the molecular pre-training representation method provided by the present application, which fuses SMILES sequence and molecular graph;

[0037] Figure 3 is one of the flowcharts of the molecular pre-training representation method provided by the present application, which fuses SMILES sequence and molecular graph;

[0038] Figure 4 is one of the flowcharts of the molecular pre-training representation method provided by the present application, which fuses SMILES sequence and molecular graph;

[0039] Figure 5 is one of the flowcharts of the molecular pre-training representation method provided by the present application, which fuses SMILES sequence and molecular graph;

[0040] Figure 6 is one of the flowcharts of the molecular pre-training representation method provided by the present application, which fuses SMILES sequence and molecular graph;

[0041] Figure 7 is a structural schematic diagram of an electronic device provided by the present application.

[0042] Reference signs:

[0043] 110: data acquisition module; 120: processing module; 130: molecular representation module;

[0044] 710: processor; 720: communication interface; 730: memory; 740: communication bus. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0046] The present application will be described below in combination with Figures 1-5 A molecular pre-training representation method fusing SMILES sequence and molecular graph is described in the present application, comprising:

[0047] S100, obtaining a character sequence in SMILES form and a molecular graph;

[0048] S200, inputting the character sequence in SMILES form and the molecular graph into a pre-trained Transformer model for processing from the molecular level and the fragment level respectively, and outputting molecular representation vectors;

[0049] S300, taking the molecular representation vectors as input of a downstream task to complete molecular representation.

[0050] In the present application, a single tower model (sharing a Transformer skeleton) is used, and different pre-training tasks at the molecular fragment level and the whole molecule level are designed to fully exploit the correspondence of SMILES and molecular graph at different levels and enhance molecular representation.

[0051] Inputting the character sequence in SMILES form and the molecular graph into a pre-trained Transformer model for processing, specifically comprising:

[0052] S101, at the molecular level, the character sequence in SMILES form is subjected to a word segmenter and a linear embedding layer to obtain a first embedding vector of each character;

[0053] S102, the molecular graph is subjected to a graph neural network to obtain a second embedding vector of each node;

[0054] S103, inputting the first embedding vector and the second embedding vector into a Transformer model for processing.

[0055] Due to the sequential format of SMILES strings, NLP-based pre-training methods have recently been applied to SMILES with competitive results. For example, SMILES-BERT and ChemBERT use BERT and RoBERTa architectures to perform MLM on SMILES, respectively. Meanwhile, SMILES-Transformer and X-MOL utilize an encoder-decoder architecture to generate learned representations from SMILES. Thanks to the successful use of large-scale language models, these models are capable of pre-training on massive molecular data, such as the 1.1 billion molecules in X-MOL. However, previous work has shown that these methods are limited in capturing molecular structure.

[0056] Generally, pre-training methods based on molecular graphs can be categorized into three main types: generative methods, predictive methods, and contrastive methods. Generative methods aim to reconstruct the original molecular graph. For example, they generate a topic tree for the molecule using a predefined vocabulary. Predictive methods construct context based on the graph's structure and employ context-aware masks for self-supervised learning. For example, neighbors in the graph are defined as context, and relevant context on the molecular graph is used to predict masked atoms / edges / motivities. Contrastive methods primarily apply contrastive learning to the molecular graph. For example, the graph is first transformed by masking atoms / edges / subgraphs, positive and negative pairs are constructed, and constant loss is utilized. Furthermore, some works use chemical knowledge for data augmentation in contrastive learning.

[0057] Because a molecule can be represented in multiple forms—for example, SMILES, molecular maps, IUPAC, and cell-based microscopy images—several hybrid approaches have been proposed to combine different forms to learn a unified representation. Examples include combining molecular maps with SMILES, IUPAC with SMILES, and cell-based microscopy images with molecular maps. However, these methods all use a two-stream model architecture, where a separate encoder is first employed to obtain a representation for each data form, and then further losses are used to align these representations, such as visual... Figure 1 Consistency (in DMP), multilingual contrastive alignment (in MM-Deacon), and InfoLOOB contrastive loss (in CLOOME). These two-stream models excel at learning representations for each form but struggle to capture the rich alignment information between different forms.

[0058] The present application adapts to the corresponding relationship between SMILES and molecular graphs. The basic idea of fine-grained alignment is designed for the framework of SMILES and molecular graphs, which is also applicable to the integration of other data forms, such as IUPAC and cell-based microscope images. In the future, we will study how to design a general framework to integrate different data forms. Our work is also inspired by some recent visual-linguistic multimodal studies, which can be divided into dual-stream and single-stream methods. Dual-stream methods such as CLIP, ALIGN and WenLan are good at capturing weak correlations between vision and language, while single-stream methods such as Oscar are more suitable for building strong correlation models. SMILES and molecular graphs have typical strong correlation, that is, SMILES provides detailed molecular graph description, just like the strong correlation between image and language in image captioning tasks, so the single-stream structure is more suitable for the fusion of different data forms of molecules.

[0059] In the present application, the first embedding vector and the second embedding vector in the embedding layer are intended to map the given SMILES and molecular graph into a vector for further calculation. For the input SMILES, a word-based token editor is used to parse it into a series of tokens S = [t1, t2, …, t n ], and then an embedding layer is applied to obtain the embedding of SMILES, denoted as s = [s1, s2, …, s n ], where s i is the token i embedding with size D. The input graph is represented as G = {V, E}, where V is the vertex set, v i ∈V, indicating the i-th atom, E is the edge set, e ij ∈E, representing the edge between the i-th atom and the j-th atom. The embedding of the graph g = [g1, g2, …, gm] m ] is obtained, where m = |V| is the number of atoms, and g i is the D-dimensional vector of the i-th atom.

[0060] The character sequence in the form of SMILES and the molecular graph are input into the pre-trained Transformer model for processing, which further comprises:

[0061] S201, at the segment level, the BRICS algorithm is used to segment the molecular graph, and the segmented molecular graph is obtained;

[0062] S202, the character sequence in the form of SMILES is labeled by labeling rules to obtain a character sequence with labels.

[0063] The Transformer layer, after the embedding layer, obtains the SMILES embedding and the position embedding p sand connect this sum with the graph embedding, then use a Transformer-based encoder to perform multimodal fusion, including inter- and intra-modal fusion. To this end, obtain an embedding x = θ(s + p s ,g) for each molecule, where x = (x1,x2,…,x n ,x n+1 ,…,x n+m ),

[0064] where the first n elements and the last m elements of x represent the corresponding SMILES and molecular graph representation, respectively. Then, apply a typical average set operation to analyze x to obtain the final molecular description, denoted as x cls .

[0065]

[0066] Based on x cls Different losses can be designed to facilitate the pre-training process. Considering the characteristics of the SMILE and molecular graph relationship, local (i.e., fragment-level) and global (i.e., molecule-level) losses are designed.

[0067] The idea of the molecular picture segment decomposition algorithm is to first apply a mature method to decompose the graph into different segments, and then assign each SMILES character to the corresponding segment according to the SMILES definition.

[0068] Specifically, using the BRICS algorithm, the graph is segmented into different segments, i.e., kj segments.

[0069] For convenience, the data obtained for the jth molecule is denoted as K. The label of a node can represent two vectors to prove which fragment the atom belongs to, where l g represents the segment, and the value of i's ID the first atom is from 0 to K-1.

[0070] The tasks at the fragment level include: cross-modal masking and fragment alignment;

[0071] S301, the cross-modal mask performs masking at the fragment level and the character level, respectively;

[0072] S302, the fragment-level mask needs to provide complete information of the other modality;

[0073] S303, the character-level mask only needs to provide a single modality.

[0074] The multi-level cross-modal mask uses the entire information of the adjacent text and other modalities to recover the corresponding masked SMILES or molecular picture segment.

[0075] The target is defined as predicting the masked SMILES or molecular graph token using all the information of its surrounding context and other ways of negative samples, with modal emphasis. In this way, both inter-modal and intra-modal modalities can be considered. Because the token is relatively small compared to the fragment, the contextual information in the same modality is sufficient for MLM. These masking strategies are compared in the ablation study.

[0076] First, character-level masking is introduced. For a given SMILES-molecular graph pair (S, G), a negative pair (S, G') is constructed by randomly replacing graph G with another molecular graph G' in the training set. Then, the input tokens of SMILES and the atoms of graph are independently masked with the same ratio r. Specifically, for SMILES, |S|*rt is randomly selected from S for masking, and the type of the masked token is predicted.

[0077] |V|*rt atoms are randomly selected for the molecular graph, and the initial features of the two atoms are set. The selected atoms and their adjacent edges are connected to the pre-defined masked atoms, and then GNN is used to obtain node embeddings on the masked graph. The context information of the masked atom is predicted in the manner of Grover, and the total loss is defined as follows:

[0078]

[0079] S \m , G' \m represent the context in SMILES and graph G. For fragment-level cross-modal masking (f-CMM), we randomly select rr*K fragments, and then randomly select a modality to mask the selected fragments with a probability of 0.5. The fragment-level mask, except that the fragment is the masking unit, the data processing and prediction target are the same as the symbol-level masking. Therefore, the loss can be written as:

[0080]

[0081] where s m and g m represent the masked SMILES and graph fragments S \m , G \m represent the context in S and G.

[0082] Fragment alignment compares the SMILES fragment representation after multi-head attention mechanism with the average-pooled molecular picture fragment representation for contrastive learning.

[0083] The fine-grained correspondence between modalities is captured by the Transformer model.

[0084] The fragment alignment (FLA) in the application is a fine-grained cross-modal alignment strategy. The basic idea is to obtain the corresponding fragment representations in the SMILES and molecular graph pairs, and then align them through a contrastive loss. For each fragment The representation of each fragment is obtained by a multi-head attention layer; while the representation of the graph is a mean set directly obtained by aggregating the relevant atom embeddings. Obviously and form a positive pair. Because they represent the same fragment in different modalities. While the negative pair is constructed as follows. For each fragment The negative picture segment set Ns includes other fragments of the same graph and all fragments of other molecular graphs. Here, the same fragment in different graphs is considered to form a negative pair because the context information is different. Similarly, the corresponding SMILES fragment negative example set Ng can be obtained for each fragment The loss is a combination of the SMILES-based and graph-based contrastive losses, i.e. LFLA=Ls+Lg, where

[0085]

[0086]

[0087] where cos(,) represents the cosine similarity function, and τ represents the temperature hyperparameter.

[0088] The molecular representation vector is input as the input of the downstream task to complete the molecular representation, specifically including:

[0089] S401, molecular level task, predicting the positive and negative pairs of character sequences-forms-molecular graph, and judging whether they come from the same molecule;

[0090] S402, making predictions on molecular fingerprints and functional group information through professional knowledge learning, and strengthening the learning of chemical professional knowledge;

[0091] S403, after pre-training, the Transformer model learns the cross-modal information of the fragment and molecule levels at the same time, and enhances the molecular representation.

[0092] In the application, eight downstream tasks of molecular property prediction and drug-drug interaction prediction are tested, and compared with a variety of baseline models, and the results are significantly improved in six tasks.

[0093] In terms of molecular property prediction, the scheme selects 4 classification tasks (BBBP, SIDER, ClinTox, HIV) and 3 regression tasks (FreeSolv, ESOL, Lipophilicity) from MoleculeNet. It can be seen that in comparison with supervised methods, SMILES methods, molecular graph methods and hybrid methods, the scheme performs best on 3 regression tasks and best or suboptimal on classification tasks.

[0094] In molecular property prediction and drug-drug interaction prediction, a total of 8 downstream tasks are tested and compared with a variety of baseline models, and the results are significantly improved in 6 tasks.

[0095] Referring to Tables 1 and 2, in terms of molecular property prediction, the scheme selects 4 classification tasks (BBBP, SIDER, ClinTox, HIV) and 3 regression tasks (FreeSolv, ESOL, Lipophilicity) from MoleculeNet. It can be seen that in comparison with supervised methods, SMILES methods, molecular graph methods and hybrid methods, the scheme performs best on 3 regression tasks and best or suboptimal on classification tasks.

[0096] Table 1 Molecular property prediction results

[0097]

[0098] In Table 1, the bolded numbers are the optimal results, and the underlined numbers are the suboptimal results.

[0099] Table 2 Drug-drug interaction prediction results

[0100]

[0101] In Table 2, the bolded numbers are the optimal results, and the underlined numbers are the suboptimal results.

[0102] The application provides a molecular pre-training representation method fusing SMILES sequences and molecular graphs. The Transformer model breaks through the limitations of existing hybrid methods and captures fine-grained cross-modal semantics between SMILES sequences and molecular graphs. Obtain corresponding SMILES and molecular picture segments, which are used as heterogeneous inputs in the pre-training task together with the labels; and use a shared transformer for deep cross-modal fusion. By introducing two new segment-level pre-training tasks, namely multi-level cross-modal masking and segment-level alignment, the fine-grained correspondence between the modes is captured, the correspondence between SMILES and molecular graphs at different levels is fully explored, and the molecular representation is enhanced.

[0103] ReferenceFigure 6 The application also discloses a molecular pre-training representation system fusing a SMILES sequence and a molecular graph, and the system comprises:

[0104] a data acquisition module 110, which is configured to acquire a character sequence in a SMILES form and a molecular graph;

[0105] a processing module 120, which is configured to input the character sequence in the SMILES form and the molecular graph into a pre-trained Transformer model for processing at a molecular level and a fragment level respectively, and output a molecular representation vector;

[0106] a molecular representation module 130, which is configured to input the molecular representation vector as an input of a downstream task and complete molecular representation.

[0107] In the processing module, at the molecular level, the character sequence in the SMILES form is subjected to a tokenizer and a linear embedding layer to obtain a first embedding vector of each character.

[0108] The molecular graph is subjected to a graph neural network to obtain a second embedding vector of each node.

[0109] The first embedding vector and the second embedding vector are jointly input into the Transformer model for processing.

[0110] At the fragment level, the BRICS algorithm is used to divide the molecular graph into divided molecular image fragments.

[0111] The character sequence in the SMILES form is labeled by a labeling rule to obtain a character sequence with labels.

[0112] The task at the fragment level comprises cross-modal masking and fragment alignment.

[0113] The cross-modal masking is performed at a fragment level and a character level.

[0114] The fragment level masking needs to provide complete information of another modality.

[0115] The character level masking only needs to provide a single modality.

[0116] The fragment alignment performs contrastive learning on a SMILES fragment representation subjected to a multi-head attention mechanism and an average-pooled molecular image fragment representation.

[0117] The Transformer model is used to capture fine-grained correspondence between modalities.

[0118] The molecular representation module inputs the molecular representation vector as an input of a downstream task to complete molecular representation, and specifically comprises:

[0119] Molecular level task, predict the form of character sequence-molecule graph positive and negative pairs, judge whether they come from the same molecule;

[0120] Through professional knowledge learning, make predictions on molecular fingerprints and functional group information, and strengthen the learning of chemical professional knowledge;

[0121] After pre-training, the Transformer model learns cross-modal information at the fragment and molecule levels at the same time, and enhances the molecular representation.

[0122] The application provides a molecular pre-training representation system fusing SMILES sequence and molecular graph, which breaks through the limitation of the existing hybrid method through the Transformer model, and captures the fine-grained cross-modal semantics between the SMILES sequence and the molecular graph. Obtain the corresponding SMILES and molecular picture segment, these segments and labels are used as heterogeneous input in the pre-training task; and a shared transformer is used for deep cross-modal fusion. By introducing two new pre-training tasks at the fragment level, namely multi-level cross-modal masking and alignment at the fragment level, the fine-grained corresponding relationship between the modes is captured, the corresponding relationship between the SMILES and the molecular graph at different levels is fully mined, and the molecular representation is enhanced.

[0123] Figure 7 An example of an entity structure diagram of an electronic device is shown as Figure 7 As shown, the electronic device can include a processor 710, a communications interface 720, a memory 730 and a communications bus 740, wherein the processor 710, the communications interface 720 and the memory 730 complete mutual communication through the communications bus 740. The processor 710 can call the logic instructions in the memory 730 to execute a molecular pre-training representation method fusing SMILES sequence and molecular graph, which comprises: acquiring a character sequence in SMILES form and a molecular graph;

[0124] Respectively from the molecular level and the fragment level, the SMILES form character sequence and the molecular graph are input into the pre-trained Transformer model for processing, and a molecular representation vector is output;

[0125] The molecular representation vector is used as the input of the downstream task to complete the molecular representation.

[0126] In addition, the logic instructions in the memory 730 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or parts of the prior art that make essential contributions or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0127] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to perform a molecular pre-training representation method of fusing SMILES sequence and molecular graph, the method comprising: obtaining a character sequence in SMILES form and a molecular graph;

[0128] respectively from the molecular level and the fragment level, inputting the character sequence in SMILES form and the molecular graph into a pre-trained Transformer model for processing, and outputting a molecular representation vector;

[0129] using the molecular representation vector as input of a downstream task to complete molecular representation.

[0130] In another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement a molecular pre-training representation method of fusing SMILES sequence and molecular graph, the method comprising: obtaining a character sequence in SMILES form and a molecular graph;

[0131] respectively from the molecular level and the fragment level, inputting the character sequence in SMILES form and the molecular graph into a pre-trained Transformer model for processing, and outputting a molecular representation vector;

[0132] using the molecular representation vector as input of a downstream task to complete molecular representation.

[0133] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0134] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0135] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A molecular pre-training representation method integrating SMILES sequences and molecular diagrams, characterized in that, include: Obtain character sequences and molecular diagrams in SMILES format; The character sequence in the form of SMILES and the molecular graph are input into a pre-trained Transformer model for processing at both the molecular and fragment levels, and the molecular representation vector is output. The molecular representation vector is used as input to the downstream task to complete the molecular representation; The character sequence in SMILES form and the molecular graph are input into a pre-trained Transformer model for processing, specifically including: At the molecular level, the character sequence in the form of SMILES is processed by a word segmenter and a linear embedding layer to obtain the first embedding vector for each character; The molecular graph is processed by a graph neural network to obtain a second embedding vector for each node; the first and second embedding vectors are then input into the Transformer model for processing. At the fragment level, the BRICS algorithm is used to divide the molecular graph into fragments to obtain fragmented molecular image segments; the character sequences in the form of SMILES are labeled according to the labeling rules to obtain labeled character sequences.

2. The molecular pre-training representation method fusing SMILES sequences and molecular maps according to claim 1, characterized in that, The fragment-level tasks include: cross-modal masking and fragment alignment; The cross-modal masking is performed at both the fragment level and the character level. The segment horizontal mask needs to provide complete information about another modality; The character horizontal mask only needs to provide a single modality.

3. The molecular pre-training representation method fusing SMILES sequences and molecular maps according to claim 2, characterized in that, The fragment alignment is learned by comparing the SMILES fragment representation after multi-head attention mechanism with the molecular image fragment representation after average pooling. The Transformer model captures fine-grained correspondences between modes.

4. The molecular pre-training representation method fusing SMILES sequences and molecular maps according to claim 2, characterized in that, The step of using the molecular representation vector as input to downstream tasks to complete molecular representation specifically includes: At the molecular level, the task is to predict positive and negative pairs of character sequences and molecular graphs to determine whether they originate from the same molecule. By learning professional knowledge, we can make predictions about molecular fingerprints and functional group information, and strengthen our knowledge of chemistry. After pre-training, the Transformer model learns cross-modal information at both the fragment and molecular levels, thereby enhancing molecular representation.

5. A molecular pre-trained representation system integrating SMILES sequences and molecular diagrams, characterized in that, The system includes: The data acquisition module is used to acquire character sequences and molecular diagrams in SMILES format; The processing module is used to input the character sequence in the form of SMILES and the molecular diagram into the pre-trained Transformer model for processing at the molecular level and the fragment level, respectively, and output the molecular representation vector. The molecular representation module is used to take the molecular representation vector as input to the downstream task to complete the molecular representation; The processing module is specifically used for: At the molecular level, the character sequence in the form of SMILES is processed by a word segmenter and a linear embedding layer to obtain the first embedding vector of each character; the molecular graph is processed by a graph neural network to obtain the second embedding vector of each node; the first and second embedding vectors are input together into the Transformer model for processing. At the fragment level, the BRICS algorithm is used to divide the molecular graph into fragments to obtain fragmented molecular image segments; the character sequences in the form of SMILES are labeled according to the labeling rules to obtain labeled character sequences.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the molecular pre-training representation method that integrates SMILES sequences and molecular graphs as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the molecular pre-training representation method that integrates SMILES sequences and molecular maps as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the molecular pre-training representation method that integrates SMILES sequences and molecular maps as described in any one of claims 1 to 4.

Images