Training, retrieval method and device of multi-granularity aligned text molecule retrieval model

Abstract

The present application provides a kind of multi-granularity alignment text molecular retrieval model training, retrieval method and device, the training method includes obtaining molecular isomerism graph and corresponding text description, molecular isomerism graph includes atom node, substructure node and molecule node, molecule node is connected with all substructure nodes, each substructure node is connected with the atom node included in it;Text molecular retrieval model is built, and text description is input into text molecular retrieval model to obtain word unit representation and sentence representation;Molecular isomerism graph is input into text molecular retrieval model to obtain atom representation, substructure representation and molecule representation;The alignment relationship between word unit representation and substructure representation is modeled into optimal transport problem, and the word unit representation aligned with each substructure representation is aggregated to obtain multi-word unit representation;Contrast learning loss function is used to optimize the alignment between multi-granularity, to obtain the trained text molecular retrieval model;To improve the accuracy of retrieval result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cross-modal retrieval, and particularly relates to a training method of a multi-granularity aligned text molecule retrieval model, a multi-granularity aligned text molecule retrieval method, a training device of a multi-granularity aligned text molecule retrieval model and a multi-granularity aligned text molecule retrieval device. BACKGROUND

[0002] In the related art, the existing text molecule cross-modal retrieval is mostly to represent the molecule as a one-dimensional sequence form, a two-dimensional molecule graph form or a three-dimensional molecule conformation form; wherein, in the one-dimensional molecule modeling aspect, the SMILES technology is widely used to represent the molecule sequence, and derives many pre-training models, such as KV-PLM, MolT5, MolXPT and Text+Chem T5; and the two-dimensional topological structure is to regard the atoms and chemical bonds as nodes and edges respectively, for example, MoMu and MoleculeSTM researches align the text and the molecule graph in a shared semantic space through cross-modal contrast learning; in addition, AMAN effectively aligns the two modalities through adversarial learning; on the basis of these methods, some methods further combine additional modalities to assist the alignment between the text and the molecule, such as MolFM combines a knowledge graph, and GIT-Mol combines an image modality; but the above methods all only focus on the global molecule information, and often ignore the fine-grained information at the substructure and atom level, while specific atoms are combined into substructures (such as hydroxyl groups) through chemical bonds, multiple substructures are connected to form a molecule, and the properties of the molecule are related to the atom and substructure information included, and ignoring the atom and substructure information inside the molecule will affect the accuracy of the retrieval result. SUMMARY

[0003] The present application aims to at least solve one of the above technical problems in the art to some extent. To this end, one object of the present application is to propose a training method of a multi-granularity aligned text molecule retrieval model, to obtain the representation of atoms, substructures and molecules through constructing a hierarchical molecule heterogeneous graph, to aggregate the representation of multiple word pieces aligned with the same substructure through optimal transport technology, and to align the text and molecule modalities at the word piece-atom, multi-word piece-substructure and sentence-molecule granularities through contrast learning, thereby improving the accuracy of the retrieval result.

[0004] A second object of the present application is to propose a multi-granularity aligned text molecule retrieval method.

[0005] A third object of the present application is to propose a training device of a multi-granularity aligned text molecule retrieval model.

[0006] A fourth object of the present application is to propose a multi-granularity aligned text molecule retrieval device.

[0007] To achieve the above object, the first aspect of the present application provides a training method of a multi-granularity aligned text molecule retrieval model, comprising the following steps: obtaining a molecule isomerism graph and a text description corresponding to each molecule isomerism graph, wherein the molecule isomerism graph comprises atom nodes, substructure nodes and molecule nodes, the molecule nodes are connected with all the substructure nodes, and each substructure node is connected with the atom nodes included therein; constructing a text molecule retrieval model, wherein the text molecule retrieval model comprises a text encoder and a molecule encoder; inputting the text description corresponding to each molecule isomerism graph into the text encoder for coding to obtain token representations and sentence representations; inputting the molecule isomerism graph into the molecule encoder for coding to obtain atom representations, substructure representations and molecule representations; modeling an alignment relationship between the token representations and the substructure representations into an optimal transport problem, and aggregating the token representations aligned with each substructure representation to obtain multi-token representations; and optimizing the alignment between the tokens and the atoms, the alignment between the multi-tokens and the substructures, and the alignment between the sentences and the molecules by using a contrastive learning loss function to obtain a trained text molecule retrieval model.

[0008] According to the training method of the multi-granularity aligned text molecule retrieval model provided by the embodiments of the present application, first, a molecule isomerism graph and a text description corresponding to each molecule isomerism graph are obtained, wherein the molecule isomerism graph comprises atom nodes, substructure nodes and molecule nodes, the molecule nodes are connected with all the substructure nodes, and each substructure node is connected with the atom nodes included therein; a text molecule retrieval model is constructed, wherein the text molecule retrieval model comprises a text encoder and a molecule encoder; each molecule isomerism graph corresponding text description is input into the text encoder for coding to obtain token representations and sentence representations; the molecule isomerism graph is input into the molecule encoder for coding to obtain atom representations, substructure representations and molecule representations; the alignment relationship between the token representations and the substructure representations is modeled into an optimal transport problem, and the token representations aligned with each substructure representation are aggregated to obtain multi-token representations; and the alignment between the tokens and the atoms, the alignment between the multi-tokens and the substructures, and the alignment between the sentences and the molecules are optimized by using a contrastive learning loss function to obtain a trained text molecule retrieval model; thereby, the representations of atoms, substructures and molecules are obtained by constructing a hierarchical molecule isomerism graph, the representations of multiple tokens aligned with the same substructure are aggregated by using an optimal transport technology, and the text and molecule modalities are aligned at the token-atom, multi-token-substructure and sentence-molecule granularity by using contrastive learning, so as to improve the accuracy of the retrieval results.

[0009] In addition, the training method of the multi-granularity aligned text molecule retrieval model provided by the above embodiments of the present application can have the following additional technical features:

[0010] Optionally, the text description corresponding to each molecular isomerism graph is input into the text encoder for encoding to obtain token representations and sentence representations, including: adding a mark at the beginning of the text description corresponding to each molecular isomerism graph to learn a sentence representation, and encoding the text description using the text encoder to obtain token representations and sentence representations.

[0011] Optionally, the alignment relationship between the token representations and the substructure representations is modeled as an optimal transport problem, and the token representations aligned with each substructure representation are aggregated to obtain multi-token representations, including: taking the token representations and the substructure representations as two independent distributions, and taking the cosine distance as the transport cost between the two representations; obtaining an optimal transport scheme with the minimum transport cost by using the non-exact nearest neighbor method of optimal transport; and obtaining the alignment relationship between the token representations and the substructure representations according to the optimal transport scheme with the minimum transport cost, so as to aggregate the token representations aligned with each substructure representation to obtain multi-token representations.

[0012] Optionally, a three-granularity contrastive learning loss function is constructed to narrow the two modalities through contrastive learning at the token-atom level, the multi-token-substructure level, and the sentence-molecule level, wherein the contrastive learning loss function adopts a classification cross-entropy loss function to maximize the similarity score between matched text-molecule pairs and minimize the score between unmatched text-molecule pairs.

[0013] To achieve the above object, the second aspect embodiment of the present application proposes a multi-granularity aligned text molecule retrieval method, including the following steps: obtaining first data to be retrieved; inputting the first data to be retrieved into a trained text molecule retrieval model to calculate the similarity between the two modalities of the first data to be retrieved and second data at the token-atom level, the multi-token-substructure level, and the sentence-molecule level, and obtaining the second data corresponding to the highest similarity of the first data to be retrieved to obtain the corresponding retrieval result, wherein the trained text molecule retrieval model is trained by using the training method according to any one of the first aspect.

[0014] To achieve the above object, the third aspect of the present application provides a device for training a multi-granularity alignment text molecule retrieval model, comprising a first acquisition module configured to acquire a molecule isomerism graph and a text description corresponding to each molecule isomerism graph, wherein the molecule isomerism graph comprises atom nodes, substructure nodes and molecule nodes, the molecule nodes are connected with all the substructure nodes, and each substructure node is connected with the atom nodes included therein; a model construction module configured to construct a text molecule retrieval model, wherein the text molecule retrieval model comprises a text encoder and a molecule encoder; a first encoding module configured to input the text description corresponding to each molecule isomerism graph into the text encoder for encoding to obtain token representations and sentence representations; a second encoding module configured to input the molecule isomerism graph into the molecule encoder for encoding to obtain atom representations, substructure representations and molecule representations; an aggregation processing module configured to model an alignment relationship between the token representations and the substructure representations into an optimal transport problem and aggregate the token representations aligned with each substructure representation to obtain multi-token representations; and an alignment optimization module configured to optimize the alignment between the tokens and the atoms, the alignment between the multi-tokens and the substructures, and the alignment between the sentences and the molecules by using a contrastive learning loss function to obtain a trained text molecule retrieval model.

[0015] In addition, the device for training a multi-granularity alignment text molecule retrieval model according to the above-mentioned embodiments of the present application can further have the following additional technical features:

[0016] Optionally, the first encoding module is further configured to add a mark at the beginning of the text description corresponding to each molecule isomerism graph to learn the sentence representations, and encode the text description by using the text encoder to obtain the token representations and the sentence representations.

[0017] Optionally, the aggregation processing module is further configured to take the token representations and the substructure representations as two independent distributions, take a cosine distance as a transport cost between the two representations, obtain an optimal transport scheme with the minimum transport cost by using an optimal transport inexact nearest point method, and obtain an alignment relationship between the token representations and the substructure representations according to the optimal transport scheme with the minimum transport cost so as to aggregate the token representations aligned with each substructure representation to obtain multi-token representations.

[0018] Optionally, three granularities of contrastive learning loss functions are constructed to narrow down two modalities by contrastive learning at a token-atom level, a multi-token-substructure level and a sentence-molecule level, wherein the contrastive learning loss function adopts a classification cross-entropy loss function to maximize a similarity score between matched text-molecule pairs and minimize a score between unmatched text-molecule pairs.

[0019] To achieve the above object, the fourth aspect of the present application provides a multi-granularity aligned text molecule retrieval device, comprising: a second acquisition module configured to acquire first data to be retrieved; and a retrieval module configured to input the first data to be retrieved into a trained text molecule retrieval model to calculate the similarity between the first data to be retrieved and second data in the token-atom level, multi-token-substructure level and sentence-molecule level by weighting, so as to obtain the second data corresponding to the highest similarity of the first data to be retrieved, and take the second data as the corresponding retrieval result, wherein the trained text molecule retrieval model is trained by using the training method according to any one of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A flowchart of a training method of a multi-granularity aligned text molecule retrieval model according to an embodiment of the present application;

[0021] Figure 2 A general network structure diagram of a text molecule retrieval model according to an embodiment of the present application;

[0022] Figure 3 A flowchart of obtaining multi-tokens by an optimal transmission non-precise nearest point method according to an embodiment of the present application;

[0023] Figure 4 A flowchart of token-atom comparison according to an embodiment of the present application;

[0024] Figure 5 A flowchart of a multi-granularity aligned text molecule retrieval method according to an embodiment of the present application;

[0025] Figure 6 A block diagram of a training device of a multi-granularity aligned text molecule retrieval model according to an embodiment of the present application;

[0026] Figure 7 A block diagram of a multi-granularity aligned text molecule retrieval device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0028] For a better understanding of the above technical solutions, the exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided so that the present application can be more thoroughly understood and the scope of the present application can be fully conveyed to those skilled in the art.

[0029] For a better understanding of the above technical solutions, the exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided so that the present application can be more thoroughly understood and the scope of the present application can be fully conveyed to those skilled in the art.

[0030] Figure 1 The flowchart of the training method of the multi-granularity aligned text molecule retrieval model according to the embodiment of the present application is shown as Figure 1 The training method of the multi-granularity aligned text molecule retrieval model according to the embodiment of the present application includes the following steps:

[0031] S101, obtaining a molecule isomerism graph and a text description corresponding to each molecule isomerism graph, wherein the molecule isomerism graph includes an atom node, a substructure node and a molecule node, the molecule node is connected with all the substructure nodes, and each substructure node is connected with the atom nodes included therein.

[0032] That is, in order to effectively encode the input molecule, the molecule graph is constructed into a molecule isomerism graph, which includes three types of nodes, (1) atom node; since each molecule is composed of atoms connected by chemical bonds, each atom is taken as a separate node, which can completely capture the detailed structure information of the molecule. (2) Substructure node; in molecular chemistry, it is believed that substructure encodes rich implicit semantic information, which is crucial for understanding the characteristics of molecules described in the text, and each molecule is decomposed into multiple substructures by using the BRICS algorithm and additional chemical rules, each substructure including multiple atoms. (3) Molecule node; a global molecule node is used to learn a comprehensive molecule representation. In order to effectively capture the relationship between nodes of different granularities, two types of edges are set: (1) Substructure-atom edge; each substructure node is connected with the atom nodes included therein. (2) Molecule-substructure edge; the molecule node is connected with all the substructure nodes.

[0033] It should be noted that, in Figure 2 the molecule isomerism graph on the left, each node at the top represents an atom node, the node with a solid line in the middle is a substructure node connected with the atom nodes included therein; at the same time, the molecule node is represented by a dashed line edge connected with all the substructure nodes; such a molecule isomerism graph can enhance the information sharing between atom, substructure and molecule nodes in the neighborhood aggregation process of the graph neural network, thereby promoting multi-granularity molecule representation learning.

[0034] S102, a text molecule retrieval model is constructed, wherein the text molecule retrieval model comprises a text encoder and a molecule encoder.

[0035] It should be noted that the constructed text encoder is a text encoder based on a pre-trained SciBERT, and the molecule encoder is a molecule encoder based on a 3-layer GCN (Graph Convolutional Network). The text and molecule representations are learned at multiple granularities, respectively. The maximum text length encoded by the SciBERT model is 256, and the GCN output dimension is 300.

[0036] S103, inputting the text description corresponding to each molecule isomerism graph into the text encoder for encoding to obtain token representation and sentence representation.

[0037] That is, the SciBERT is used to encode the input text description to obtain sentence and token level representation. Compared with other pre-training models, SciBERT performs well in encoding chemical text description, which is due to pre-training on a large-scale scientific publication corpus.

[0038] As an embodiment, inputting the text description corresponding to each molecule isomerism graph into the text encoder for encoding to obtain token representation and sentence representation comprises: adding a mark at the beginning of the text description corresponding to each molecule isomerism graph to learn the sentence representation, and using the text encoder to encode the text description to obtain the token representation and the sentence representation.

[0039] As a specific embodiment, given an input text description with N t tokens, a special [CLS] mark is first connected at the beginning of the sequence, and then the sequence is input into the SciBERT encoder, wherein the learned d-dimensional representation of the [CLS] mark is used as the sentence representation h s , and the d-dimensional representation of the remaining tokens is used as the token representation.

[0040]

[0041] S104, inputting the molecule isomerism graph into the molecule encoder for encoding to obtain atom representation, substructure representation and molecule representation.

[0042] As an embodiment, a 3-layer GCN is used to learn node representation, and these node representations correspond to three granularity representations; atom representation: wherein N a represents the number of atoms; substructure representation: wherein N m represents the number of substructures; and molecule representation: h g∈ R d .

[0043] S105, modeling the alignment relationship between the token representation and the substructure representation as an optimal transport problem, and aggregating the token representations aligned with each substructure representation to obtain a multi-token representation.

[0044] As an embodiment, modeling the alignment relationship between the token representation and the substructure representation as an optimal transport problem, and aggregating the token representations aligned with each substructure representation to obtain a multi-token representation, comprises: taking the token representation and the substructure representation as two independent distributions, and taking the cosine distance as the transport cost between the two representations; obtaining an optimal transport scheme with the minimum transport cost by using the inexact nearest point method of optimal transport; obtaining the alignment relationship between the token representation and the substructure representation according to the optimal transport scheme with the minimum transport cost, so as to aggregate the token representations aligned with each substructure representation to obtain a multi-token representation.

[0045] That is, the alignment between the input token representation and the substructure representation is modeled as an optimal transport problem, as shown in Figure 3 The optimal transport problem aims to minimize the cost of transferring one distribution to another distribution.

[0046] As a specific embodiment, the token representation and the substructure representation are taken as two independent distributions, where C ij represents the transport cost from the i-th token to the j-th substructure, represents the cosine distance between the i-th token and the j-th substructure. T = {T ij}, 1≤i≤N t , 1≤j≤N m , is defined as a transport scheme, and the multi-token representation is obtained according to the following formula:

[0047]

[0048] D j = {i | a(i) = j, 1≤i≤N t}

[0049]

[0050] where, represents the Frobenius dot product, a(i) represents the best alignment substructure index, D j represents the index set, and the corresponding token is aligned with the j-th substructure; represents the average of the token representations appearing in D j to obtain their fused multi-token representation;‖(·) represents the indicator function, Hp N represents the number of multi-words. p is the number of multi-words.

[0051] S106, the alignment between words and atoms, the alignment between multi-words and substructures, and the alignment between sentences and molecules are optimized by using a contrastive learning loss function to obtain a trained text molecule retrieval model.

[0052] As an embodiment, three granularity contrastive learning loss functions are constructed to pull two modalities through contrastive learning at the word-atom level, the multi-word-substructure level, and the sentence-molecule level, respectively, wherein the contrastive learning loss function uses a classification cross-entropy loss function to maximize the similarity score between matched text-molecule pairs and minimize the score between unmatched text-molecule pairs.

[0053] As a specific embodiment, the following training objectives are defined:

[0054]

[0055] wherein, and represent the word-sentence, multi-word-substructure, and sentence-molecule alignment losses, respectively, and a and β represent coefficients that balance the influence of different losses. The following will describe the three losses in detail:

[0056] (1) Word-atom alignment loss As shown in Figure 4 , the word-atom similarity matrix between the word representation and the atom representation is calculated; then the similarity matrix is normalized by minimum-maximum normalization along the atom dimension; subsequently, the matrix is further normalized along the word dimension to obtain alignment weights for unifying the dimensions of the word and atom representations. After multiplying these alignment weights with the atom representation, the atom representation is updated to Finally, the sum-pooling is applied to reduce the dimensions of the two representations, and the final similarity at the word and atom level is calculated using them.

[0057] As a specific embodiment, in a batch of samples of size B, the similarity between samples at the word and atom level is calculated The word-atom alignment loss is defined as the average of two classification cross-entropy losses, which correspond to the text molecule and molecule text retrieval tasks, respectively:

[0058]

[0059] (2) Multi-word-substructure alignment loss In a similar manner, the similarity between samples within the same batch at the multi-token and substructure level is computed Thus, the multi-token-substructure alignment loss The formula is as follows:

[0060]

[0061] (3) Sentence-molecule alignment loss Similar to other alignment levels, contrastive learning is used to compute the CCE loss at the sentence-molecule level; specifically, the similarity between matching sentence and molecule pairs is maximized, while the similarity between non-matching sentence and molecule pairs is minimized, sentence molecule alignment loss The definition is as follows:

[0062]

[0063] wherein, represents the similarity between the sentence representation of the qth sample and the molecule representation of the Kth sample.

[0064] In summary, according to the training method of the multi-granularity aligned text molecule retrieval model of the embodiment of the present application, first, the molecular isomerism graph and the text description corresponding to each molecular isomerism graph are obtained, wherein the molecular isomerism graph includes atom nodes, substructure nodes and molecule nodes, the molecule nodes are connected with all the substructure nodes, and each substructure node is connected with the atom nodes included therein; a text molecule retrieval model is constructed, wherein the text molecule retrieval model includes a text encoder and a molecule encoder; the text description corresponding to each molecular isomerism graph is input to the text encoder for encoding to obtain token representation and sentence representation; the molecular isomerism graph is input to the molecule encoder for encoding to obtain atom representation, substructure representation and molecule representation; the alignment relationship between the token representation and the substructure representation is modeled as an optimal transport problem, and the token representation aligned with each substructure representation is aggregated to obtain multi-token representation; the alignment between the token and the atom, the alignment between the multi-token and the substructure, and the alignment between the sentence and the molecule are optimized by using a contrastive learning loss function to obtain a trained text molecule retrieval model; thereby, the representations of atoms, substructures and molecules are obtained by constructing a hierarchical molecular isomerism graph, the representations of multiple tokens aligned with the same substructure are aggregated by optimal transport technology, and the text and molecule modalities are aligned at the token-atom, multi-token-substructure and sentence-molecule granularities by contrastive learning, thereby improving the accuracy of the retrieval results.

[0065] In order to realize the above-mentioned embodiment, the embodiment further provides a multi-granularity aligned text molecule retrieval method.

[0066] Figure 5A flowchart of a multi-granularity aligned text molecule retrieval method according to an embodiment of the present application is shown in FIG. 1, which includes the following steps: Figure 5

[0067] In S201, first data to be retrieved is obtained.

[0068] In S202, the first data to be retrieved is input into a trained text molecule retrieval model to calculate the similarity between the two modalities at the token-atom level, the multi-token-substructure level, and the sentence-molecule level by weighting, so as to obtain the second data with the highest similarity corresponding to the first data to be retrieved, and obtain the corresponding retrieval result. The trained text molecule retrieval model is trained by using the training method described above.

[0069] The multi-granularity aligned text molecule retrieval method according to the embodiments of the present application calculates the similarity between the two modalities at three levels by weighting in the reasoning process of text molecule retrieval, and retrieves the molecule with the highest similarity. In the molecule text retrieval task, the process is performed in the opposite direction.

[0070] In order to implement the above-mentioned embodiments, the embodiments of the present application provide a training device of a multi-granularity aligned text molecule retrieval model.

[0071] Figure 6 FIG. 2 is a block diagram of a training device of a multi-granularity aligned text molecule retrieval model according to an exemplary embodiment. Referring to FIG. 2, the device includes a first obtaining module 1, a model constructing module 2, a first encoding module 3, a second encoding module 4, an aggregation processing module 5, and an alignment optimization module 6. Figure 6

[0072] The first obtaining module 1 is configured to obtain a molecule isomerism graph and a text description corresponding to each molecule isomerism graph. The molecule isomerism graph includes atom nodes, substructure nodes, and molecule nodes. The molecule nodes are connected to all the substructure nodes, and each substructure node is connected to the atom nodes included therein. The model constructing module 2 is configured to construct a text molecule retrieval model. The text molecule retrieval model includes a text encoder and a molecule encoder. The first encoding module 3 is configured to input the text description corresponding to each molecule isomerism graph into the text encoder for encoding to obtain token representation and sentence representation. The second encoding module 4 is configured to input the molecule isomerism graph into the molecule encoder for encoding to obtain atom representation, substructure representation, and molecule representation. The aggregation processing module 5 is configured to model the alignment relationship between the token representation and the substructure representation as an optimal transport problem, and aggregate the token representation aligned with each substructure representation to obtain multi-token representation. The alignment optimization module 6 is configured to optimize the alignment between the token and the atom, the alignment between the multi-token and the substructure, and the alignment between the sentence and the molecule by using a contrastive learning loss function, so as to obtain a trained text molecule retrieval model.​​

[0073] As an embodiment, the first encoding module 3 is further configured to mark the beginning of each molecular isomerism graph described in the text to learn the sentence representation, encode the text description using a text encoder to obtain the token representation and the sentence representation.

[0074] As an embodiment, the aggregation processing module 5 is further configured to take the token representation and the substructure representation as two independent distributions, and take the cosine distance as the transmission cost between the two representations; obtain an optimal transmission scheme with the minimum transmission cost by using an inexact nearest point method of optimal transmission; and obtain an alignment relationship between the token representation and the substructure representation according to the optimal transmission scheme with the minimum transmission cost, so as to aggregate the token representation aligned with each substructure representation to obtain a multi-token representation.

[0075] As an embodiment, a three-granularity contrastive learning loss function is constructed to narrow the two modalities by contrastive learning at the token-atom level, the multi-token-substructure level, and the sentence-molecule level, wherein the contrastive learning loss function adopts a classification cross-entropy loss function to maximize the similarity score between matched text-molecule pairs and minimize the score between unmatched text-molecule pairs.

[0076] As to the apparatus in the above-mentioned embodiments, the specific manner in which each module performs the operation has been described in detail in the embodiments of the training method of the multi-granularity aligned text molecule retrieval model, and will not be described in detail here.

[0077] In order to implement the above-mentioned embodiments, the embodiments of the present disclosure provide a multi-granularity aligned text molecule retrieval apparatus.

[0078] Figure 7 is a block diagram of a multi-granularity aligned text molecule retrieval apparatus according to an exemplary embodiment. Referring to Figure 7 The apparatus comprises a second acquisition module 10 and a retrieval module 20.

[0079] The second acquisition module 10 is configured to acquire the first data to be retrieved; and the retrieval module 20 is configured to input the first data to be retrieved into the trained text molecule retrieval model to calculate the similarity between the two modalities of the first data to be retrieved and the second data at the token-atom level, the multi-token-substructure level, and the sentence-molecule level, so as to obtain the second data corresponding to the highest similarity of the first data to be retrieved, and take the second data as the corresponding retrieval result, wherein the trained text molecule retrieval model is trained by using the training method as described above.

[0080] With respect to the apparatus in the above-described embodiments, in which the specific manner in which the various means perform operations has been described in detail in the embodiments of the text molecule retrieval method related to the multi-granularity alignment, no detailed elaboration will be made here.

[0081] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, systems, or computer program products. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code.

[0082] The present application is described in reference to the flowcharts and / or block diagrams of the methods, apparatus (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce an apparatus that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flowcharts and / or block diagrams.

[0083] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flowcharts and / or block diagrams.

[0084] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flowcharts and / or block diagrams.

[0085] It is to be noticed that the term "comprising", used in the claims, should not be interpreted as being restricted to the means listed thereafter; it does not exclude other elements or steps. It is thus to be interpreted as specifying the presence of the stated features but not precluding the presence of further or additional features in other claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. It is further to be noted that claims, which do not include the word "means" cannot be interpreted under 35 U.S.C. 112, sixth paragraph. The application can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unitary claim, the features of the different embodiments can be combined with each other if this is explicit or implicit to persons in the art from the context. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these

[0086] Although the preferred embodiments of the application have been described, those skilled in the art will understand that they can be subjected to many changes and modifications without departing from the spirit and scope of the application. Accordingly, it is intended to include all such changes and modifications in so far as they come within the scope of the application taken in conjunction with the above description.

[0087] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

[0088] In the description of the application, it is to be understood that the terms "first", "second", "third" and the like, merely identify names of particular features described and do not necessarily limit the features to the number named. The reference numerals in the claims denote the several aspects of the application and are not meant to be a limitation thereof.

[0089] In the present application, unless otherwise specifically noted, the terms "mounting", "connecting", "connecting", "fixing" and the like, should be understood in a broad sense and for example, can mean fixed connection, detachable connection, or integral connection; can mean mechanical connection, or electrical connection; can mean direct connection, or indirect connection via an intermediate medium; can mean internal connection of two elements, or interaction between two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0090] In the present application, unless otherwise explicitly specified and limited, a first feature is "on" or "under" a second feature can mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, the first feature is "over", "above" and "on top of" the second feature can mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is horizontally higher than the second feature. The first feature is "under", "below" and "underneath" the second feature can mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is horizontally lower than the second feature.

[0091] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present application and the features of the different embodiments or examples without contradiction.

[0092] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-described embodiments within the scope of the present application.

Claims

1. A training method for a multi-granularity aligned text molecule retrieval model, characterized in that, Includes the following steps: Obtain molecular isomer maps and text descriptions corresponding to each molecular isomer map, wherein the molecular isomer map includes atomic nodes, substructure nodes and molecular nodes, the molecular nodes are connected to all substructure nodes, and each substructure node is connected to its included atomic nodes. Construct a text molecular retrieval model, wherein the text molecular retrieval model includes a text encoder and a molecular encoder; The text description corresponding to each molecular isomer diagram is input into the text encoder for encoding to obtain word representation and sentence representation; The molecular isomer map is input into the molecular encoder for encoding to obtain atomic representation, substructure representation and molecular representation; The alignment relationship between the lexical representation and the substructure representation is modeled as an optimal transmission problem, and the lexical representations aligned with each substructure representation are aggregated to obtain a multi-lexical representation; The contrastive learning loss function is used to optimize the alignment between words and atoms, the alignment between multi-words and substructures, and the alignment between sentences and molecules, so as to obtain a well-trained text molecule retrieval model. Specifically, the alignment relationship between the lexical representation and the substructure representation is modeled as an optimal transmission problem, and lexical representations aligned with each substructure representation are aggregated to obtain multi-lexical representations, including: The lexical representation and the substructure representation are treated as two independent distributions, and the cosine distance is used as the transmission cost between the two representations. The optimal transmission scheme with the minimum transmission cost is obtained by using the inaccurate nearest neighbor method for optimal transmission. The alignment relationship between the lexical representation and the substructure representation is obtained based on the optimal transmission scheme with the minimum transmission cost, so as to aggregate the lexical representations aligned with each substructure representation to obtain a multi-lexical representation.

2. The training method for the multi-granularity aligned text molecule retrieval model as described in claim 1, characterized in that, The text description corresponding to each molecular isomer diagram is input into the text encoder for encoding to obtain lexical representations and sentence representations, including: A marker is added to the beginning of the text description corresponding to each molecular isomer diagram to learn the sentence representation. The text description is then encoded using the text encoder to obtain the lexical representation and the sentence representation.

3. The training method for the multi-granularity aligned text molecule retrieval model as described in claim 1, characterized in that, We construct three contrastive learning loss functions at three granularities: word-atom level, multi-word-substructure level, and sentence-molecule level. These functions use contrastive learning to bring two modalities closer together. The contrastive learning loss function employs the classification cross-entropy loss function to maximize the similarity score between matched text-molecule pairs and minimize the similarity score between mismatched text-molecule pairs.

4. A text molecule retrieval method with multi-granularity alignment, characterized in that, Includes the following steps: Retrieve the first data to be searched; The first data to be retrieved is input into the trained text molecular retrieval model to calculate the similarity between the two modalities of the first data to be retrieved and the second data at the word-atom level, the multi-word-substructure level, and the sentence-molecule level, in order to obtain the second data with the highest similarity to the first data to be retrieved, and to obtain the corresponding retrieval result. The trained text molecular retrieval model is trained using the training method described in any one of claims 1-3.

5. A training device for a multi-granularity aligned text molecule retrieval model, characterized in that, include: The first acquisition module is used to acquire molecular isomer diagrams and text descriptions corresponding to each molecular isomer diagram. The molecular isomer diagram includes atomic nodes, substructure nodes and molecular nodes. Molecular nodes are connected to all substructure nodes, and each substructure node is connected to its included atomic nodes. A model building module is used to build a text molecular retrieval model, wherein the text molecular retrieval model includes a text encoder and a molecular encoder; The first encoding module is used to input the text description corresponding to each molecular isomer diagram into the text encoder for encoding to obtain word representation and sentence representation; The second encoding module is used to input the molecular isomer map into the molecular encoder for encoding to obtain atomic representation, substructure representation and molecular representation; An aggregation processing module is used to model the alignment relationship between the lexical representation and the substructure representation as an optimal transmission problem, and to aggregate the lexical representations aligned with each substructure representation to obtain a multi-lexical representation; The alignment optimization module is used to optimize the alignment between words and atoms, between multi-words and substructures, and between sentences and molecules using a contrastive learning loss function, so as to obtain a well-trained text molecule retrieval model. The aggregation processing module is further used for, The lexical representation and the substructure representation are treated as two independent distributions, and the cosine distance is used as the transmission cost between the two representations. The optimal transmission scheme with the minimum transmission cost is obtained by using the inaccurate nearest neighbor method for optimal transmission. The alignment relationship between the lexical representation and the substructure representation is obtained based on the optimal transmission scheme with the minimum transmission cost, so as to aggregate the lexical representations aligned with each substructure representation to obtain a multi-lexical representation.

6. The training apparatus for the multi-granularity aligned text molecule retrieval model as described in claim 5, characterized in that, The first encoding module is also used for, A marker is added to the beginning of the text description corresponding to each molecular isomer diagram to learn the sentence representation. The text description is then encoded using the text encoder to obtain the lexical representation and the sentence representation.

7. The training apparatus for the multi-granularity aligned text molecule retrieval model as described in claim 5, characterized in that, We construct three contrastive learning loss functions at three granularities: word-atom level, multi-word-substructure level, and sentence-molecule level. These functions use contrastive learning to bring two modalities closer together. The contrastive learning loss function employs the classification cross-entropy loss function to maximize the similarity score between matched text-molecule pairs and minimize the similarity score between mismatched text-molecule pairs.

8. A text molecule retrieval device with multi-granularity alignment, characterized in that, include: The second acquisition module is used to acquire the first data to be retrieved; The retrieval module is used to input the first data to be retrieved into a trained text molecular retrieval model to calculate the similarity between the two modalities of the first data to be retrieved and the second data at the word-atom level, the multi-word-substructure level, and the sentence-molecule level, in order to obtain the second data with the highest similarity to the first data to be retrieved, and to use the second data as the corresponding retrieval result. The trained text molecular retrieval model is trained using the training method described in any one of claims 1-3.

Images