Multi-granularity aligned text molecule retrieval model training method and device and multi-granularity aligned text molecule retrieval model training and retrieval method and device

Abstract

The invention provides a multi-granularity aligned text molecular retrieval model training and retrieval method and device.The training method comprises the steps that a molecular heterogeneous graph and corresponding text description are obtained, the molecular heterogeneous graph comprises atomic nodes, substructure nodes and molecular nodes, the molecular nodes are connected with all the substructure nodes, and the atomic nodes are connected with all the substructure nodes; each substructure node is connected with an atomic node included by the substructure node; constructing a text molecule retrieval model, and inputting the text description into the text molecule retrieval model to obtain lexical representation and sentence representation; inputting the molecular isomerism graph into a text molecular retrieval model to obtain atomic representation, substructure representation and molecular representation; modeling an alignment relationship between the lexical representation and the substructure representation into an optimal transmission problem, and aggregating the lexical representation aligned with each substructure representation to obtain a multi-lexical representation; adopting a contrast learning loss function to optimize alignment among multiple granularities so as to obtain a trained text molecule retrieval model; therefore, the accuracy of the retrieval result is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cross-modal retrieval, and in particular to a training method for a multi-granularity aligned text molecule retrieval model, a multi-granularity aligned text molecule retrieval method, a multi-granularity aligned text molecule retrieval model training device, and a multi-granularity aligned text molecule retrieval device. Background Art

[0002] In the related technologies, most of the existing cross-modal retrieval of text molecules represents molecules as one-dimensional sequence form, two-dimensional molecular graph form or three-dimensional molecular conformation form; among them, in the one-dimensional molecular modeling, SMILES technology is widely used to represent molecular sequences, and many pre-trained models have been derived, such as KV-PLM, MolT5, MolXPT and Text+Chem T5, etc.; the two-dimensional topological structure regards atoms and chemical bonds as nodes and edges respectively. For example, MoMu and MoleculeSTM and other studies align text and molecular graphs in a shared semantic space through cross-modal contrastive learning; in addition, AMAN uses adversarial learning to effectively align the two modalities; based on these methods, some methods further combine additional modalities to assist in the alignment between text and molecules, such as MolFM combined with knowledge graphs, and GIT-Mol combined with image modalities; however, the above methods only focus on global molecular information, and often ignore the fine-grained information at the substructure and atomic level. Specific atoms are connected and combined into substructures (such as hydroxyl groups) through chemical bonds, and multiple substructures are connected to form molecules. The properties of molecules are related to the atoms and substructure information they include. Ignoring the atoms and substructure information inside the molecule will affect the accuracy of the retrieval results. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems in the above-mentioned technology to a certain extent. To this end, one purpose of the present invention is to propose a training method for a multi-granularity aligned text molecule retrieval model, by constructing a hierarchical molecular heterogeneous graph to obtain the representation of atoms, substructures and molecules, and by aggregating the representations of multiple word units aligned with the same substructure through optimal transfer technology, and by contrastive learning, aligning text and molecular modalities at word unit-atom, multi-word unit-substructure and sentence-molecule granularity, thereby improving the accuracy of retrieval results.

[0004] The second object of the present invention is to propose a multi-granularity aligned text molecule retrieval method.

[0005] The third object of the present invention is to provide a training device for a multi-granularity aligned text molecule retrieval model.

[0006] The fourth objective of the present invention is to provide a multi-granularity aligned text molecule retrieval device.

[0007] To achieve the above-mentioned purpose, the first aspect of the present invention proposes a training method for a multi-granularity aligned text molecule retrieval model, which method includes the following steps: obtaining a molecular isomerism graph and a text description corresponding to each molecular isomerism graph, wherein the molecular isomerism graph includes atomic nodes, substructure nodes and molecular nodes, and the molecular nodes are connected to all substructure nodes, and each substructure node is connected to the atomic nodes it includes; constructing a text molecule retrieval model, wherein the text molecule retrieval model includes a text encoder and a molecular encoder; inputting the text description corresponding to each molecular isomerism graph into the text encoder for encoding to obtain a word unit representation and a sentence representation; inputting the molecular isomerism graph into the molecular encoder for encoding to obtain an atomic representation, a substructure representation and a molecular representation; modeling the alignment relationship between the word unit representation and the substructure representation as an optimal transmission problem, and aggregating the word unit representation aligned with each substructure representation to obtain a multi-word unit representation; using a contrastive learning loss function to optimize the alignment between word units and atoms, the alignment between multi-word units and substructures, and the alignment between sentences and molecules to obtain a trained text molecule retrieval model.

[0008] According to the training method of the multi-granularity aligned text molecule retrieval model of the embodiment of the present invention, first, a molecular isomerism graph and a text description corresponding to each molecular isomerism graph are obtained, wherein the molecular isomerism graph includes atomic nodes, substructure nodes and molecular nodes, the molecular nodes are connected to all substructure nodes, and each substructure node is connected to the atomic nodes it includes; a text molecule retrieval model is constructed, wherein the text molecule retrieval model includes a text encoder and a molecule encoder; the text description corresponding to each molecular isomerism graph is input into the text encoder for encoding to obtain a word unit representation and a sentence representation; the molecular isomerism graph is input into the molecule encoder for encoding to obtain an atomic representation, a substructure representation and a molecule representation; The alignment relationship between word-gram representation and substructure representation is modeled as an optimal transfer problem, and the word-gram representations aligned with each substructure representation are aggregated to obtain multi-word-gram representations; the contrastive learning loss function is used to optimize the alignment between word-grams and atoms, the alignment between multi-word-grams and substructures, and the alignment between sentences and molecules to obtain a trained text molecule retrieval model; thus, the representations of atoms, substructures, and molecules are obtained by constructing a hierarchical molecular heterogeneous graph, and the representations of multiple word-grams aligned with the same substructure are aggregated through optimal transfer technology, and through contrastive learning, the text and molecular modalities are aligned at the word-gram-atom, multi-word-gram-substructure, and sentence-molecule granularity, thereby improving the accuracy of the retrieval results.

[0009] In addition, the training method of the multi-granularity aligned text molecule retrieval model proposed in the above embodiment of the present invention may also have the following additional technical features:

[0010] Optionally, the text description corresponding to each molecular isomerization graph is input into the text encoder for encoding to obtain word unit representation and sentence representation, including: adding a mark at the beginning of the text description corresponding to each molecular isomerization graph to learn the sentence representation, and using the text encoder to encode the text description to obtain word unit representation and sentence representation.

[0011] Optionally, the alignment relationship between the word-gram representation and the substructure representation is modeled as an optimal transmission problem, and the word-gram representations aligned with each substructure representation are aggregated to obtain a multi-word-gram representation, including: treating the word-gram representation and the substructure representation as two independent distributions, and taking the cosine distance as the transmission cost between the two representations; using an inexact neighboring point method for optimal transmission to obtain an optimal transmission scheme with the minimum transmission cost; and obtaining the alignment relationship between the word-gram representation and the substructure representation based on the optimal transmission scheme with the minimum transmission cost, so as to aggregate the word-gram representations aligned with each substructure representation to obtain a multi-word-gram representation.

[0012] Optionally, three granularity contrastive learning loss functions are constructed to bring the two modalities closer through contrastive learning at the word-atom level, multi-word-substructure level and sentence-molecule level, respectively, wherein the contrastive learning loss function adopts a categorical cross entropy loss function to maximize the similarity scores between matching text-molecule pairs and minimize the scores between mismatching text-molecule pairs.

[0013] To achieve the above-mentioned purpose, the second aspect of the present invention proposes a multi-granularity aligned text molecule retrieval method, comprising the following steps: obtaining the first data to be retrieved; inputting the first data to be retrieved into a trained text molecule retrieval model to weightedly calculate the similarity between the two modes of the first data to be retrieved and the second data at the word-atom level, the multi-word-substructure level and the sentence-molecule level, so as to obtain the second data with the highest similarity to the first data to be retrieved, so as to obtain the corresponding retrieval result, wherein the trained text molecule retrieval model is trained using the training method described in any one of the first aspects.

[0014] To achieve the above-mentioned purpose, the third aspect of the present invention proposes a training device for a multi-granularity aligned text molecule retrieval model, comprising a first acquisition module for acquiring a molecular isomerization graph and a text description corresponding to each molecular isomerization graph, wherein the molecular isomerization graph comprises an atomic node, a substructure node and a molecular node, and the molecular node is connected to all the substructure nodes, and each substructure node is connected to the atomic node it comprises; a model construction module for constructing a text molecule retrieval model, wherein the text molecule retrieval model comprises a text encoder and a molecular encoder; a first encoding module for inputting the text description corresponding to each molecular isomerization graph into The text encoder performs encoding to obtain word-gram representation and sentence representation; a second encoding module is used to input the molecular isomerization graph into the molecular encoder for encoding to obtain atomic representation, substructure representation and molecular representation; an aggregation processing module is used to model the alignment relationship between the word-gram representation and the substructure representation as an optimal transmission problem, and aggregate the word-gram representation aligned with each substructure representation to obtain a multi-word-gram representation; an alignment optimization module is used to optimize the alignment between word-grams and atoms, the alignment between multi-word-grams and substructures, and the alignment between sentences and molecules by using a contrastive learning loss function to obtain a trained text molecular retrieval model.

[0015] In addition, the training device for the multi-granularity aligned text molecule retrieval model proposed in the above embodiment of the present application may also have the following additional technical features:

[0016] Optionally, the first encoding module is also used to add a mark at the beginning of the text description corresponding to each molecular isomorphic graph to learn sentence representation, and use the text encoder to encode the text description to obtain word unit representation and sentence representation.

[0017] Optionally, the aggregation processing module is also used to treat the word-gram representation and the substructure representation as two independent distributions, and use the cosine distance as the transmission cost between the two representations; adopt the inexact neighboring point method of optimal transmission to obtain the optimal transmission plan with the minimum transmission cost; obtain the alignment relationship between the word-gram representation and the substructure representation according to the optimal transmission plan with the minimum transmission cost, so as to aggregate the word-gram representation aligned with each substructure representation to obtain a multi-word-gram representation.

[0018] Optionally, three granularity contrastive learning loss functions are constructed to bring the two modalities closer through contrastive learning at the word-atom level, multi-word-substructure level and sentence-molecule level, respectively, wherein the contrastive learning loss function adopts a categorical cross entropy loss function to maximize the similarity scores between matching text-molecule pairs and minimize the scores between mismatching text-molecule pairs.

[0019] To achieve the above-mentioned purpose, the fourth aspect of the present invention proposes a multi-granularity aligned text molecule retrieval device, including: a second acquisition module, used to acquire the first data to be retrieved; a retrieval module, used to input the first data to be retrieved into a trained text molecule retrieval model, so as to weightedly calculate the similarity between the two modes of the first data to be retrieved and the second data at the word-atom level, the multi-word-substructure level and the sentence-molecule level, so as to obtain the second data with the highest similarity to the first data to be retrieved, and use the second data as the corresponding retrieval result, wherein the trained text molecule retrieval model is trained using the training method described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a flow chart of a method for training a text molecule retrieval model with multi-granularity alignment according to an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of the overall network structure of a text molecule retrieval model according to an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of a flow chart of obtaining multi-words by an inexact neighbor point method of optimal transmission according to an embodiment of the present invention;

[0023] Figure 4 is a schematic diagram of a flow chart of word-to-atom comparison according to an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of a flow chart of a multi-granularity aligned text molecule retrieval method according to an embodiment of the present invention;

[0025] Figure 6 A block diagram of a training device for a multi-granularity aligned text molecule retrieval model according to an embodiment of the present invention;

[0026] Figure 7 Schematic diagram of a block diagram of a multi-granularity aligned text molecule retrieval device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0028] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0029] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0030] Figure 1 FIG. 1 is a flow chart of a training method for a multi-granularity aligned text molecule retrieval model according to an embodiment of the present invention. Figure 1 As shown, the training method of the multi-granularity aligned text molecule retrieval model of the embodiment of the present invention includes the following steps:

[0031] S101, obtaining a molecular isomerism graph and a text description corresponding to each molecular isomerism graph, wherein the molecular isomerism graph includes atomic nodes, substructure nodes and molecular nodes, the molecular nodes are connected to all substructure nodes, and each substructure node is connected to the atomic nodes it includes.

[0032] That is, in order to effectively encode the input molecules, the molecular graph is constructed into a molecular heterogeneous graph, which includes three types of nodes: (1) Atom nodes; Since each molecule is composed of atoms connected by chemical bonds, each atom is regarded as a separate node, which can fully capture the detailed structural information of the molecule. (2) Substructure nodes; In molecular chemistry, it is believed that substructures encode rich implicit semantic information, which is crucial for understanding the molecular properties described in the article. The BRICS algorithm and additional chemical rules are used to decompose each molecule into multiple substructures, each of which includes multiple atoms. (3) Molecular nodes; Through a global molecular node, a comprehensive molecular representation is learned. In order to effectively capture the relationship between nodes of different granularity, two types of edges are set: (1) Substructure-atom edges; Each substructure node is connected to its constituent atom nodes. (2) Molecule-substructure edges; A molecule node is connected to all substructure nodes.

[0033] It should be noted that in Figure 2 In the molecular heterogeneous graph on the left, each node at the top represents an atomic node, and the nodes with solid lines in the middle are substructure nodes, which are connected to the atomic nodes they include; at the same time, the molecular nodes are represented by dotted edges and are connected to all substructure nodes; this molecular heterogeneous graph can enhance the information sharing between atoms, substructures and molecular nodes during the neighborhood aggregation process of graph neural networks, thereby promoting multi-granularity molecular representation learning.

[0034] S102, constructing a text molecule retrieval model, wherein the text molecule retrieval model includes a text encoder and a molecule encoder.

[0035] It should be noted that the constructed text encoder is a text encoder based on the pre-trained SciBERT, and the molecular encoder is a molecular encoder based on a 3-layer GCN (Graph Convolutional Network) to learn text and molecular representations at multiple granularities respectively; among them, the maximum text length encoded by the SciBERT model is 256, and the GCN output dimension is 300 dimensions.

[0036] S103, inputting the text description corresponding to each molecular isomorphic graph into a text encoder for encoding to obtain word unit representation and sentence representation.

[0037] That is, SciBERT is used to encode the input text description to obtain sentence and word-level representations; compared with other pre-trained models, SciBERT performs well in encoding chemical text descriptions, thanks to pre-training on a large-scale corpus of scientific publications.

[0038] As an embodiment, the text description corresponding to each molecular isomerization graph is input into a text encoder for encoding to obtain word unit representation and sentence representation, including: adding a mark at the beginning of the text description corresponding to each molecular isomerization graph to learn the sentence representation, and using the text encoder to encode the text description to obtain word unit representation and sentence representation.

[0039] As a specific example, given a t The input text description of words is first connected to a special [CLS] token at the beginning of the sequence. Then, the sequence is input into the SciBERT encoder, where the learned d-dimensional representation of the [CLS] token is used as the sentence representation h s , the d-dimensional representations of the remaining tokens are used as the token representation:

[0040]

[0041] S104, inputting the molecular isomorphous graph into a molecular encoder for encoding to obtain an atomic representation, a substructure representation, and a molecular representation.

[0042] As an embodiment, a 3-layer GCN is used to learn node representations, which correspond to representations of three granularities; atomic representation: Where N a Indicates the number of atoms; substructure indicates: Where N m Indicates the number of substructures; numerator: h g∈R d .

[0043] S105, modeling the alignment relationship between the word-gram representation and the substructure representation as an optimal transfer problem, and aggregating the word-gram representation aligned with each substructure representation to obtain a multi-word-gram representation.

[0044] As an embodiment, the alignment relationship between word-gram representation and substructure representation is modeled as an optimal transmission problem, and the word-gram representations aligned with each substructure representation are aggregated to obtain a multi-word-gram representation, including: treating the word-gram representation and the substructure representation as two independent distributions, and taking the cosine distance as the transmission cost between the two representations; using the inexact neighbor point method of optimal transmission to obtain an optimal transmission scheme with the minimum transmission cost; and obtaining the alignment relationship between the word-gram representation and the substructure representation according to the optimal transmission scheme with the minimum transmission cost, so as to aggregate the word-gram representations aligned with each substructure representation to obtain a multi-word-gram representation.

[0045] That is, the alignment between the input token representation and the substructure representation is modeled as an optimal transfer problem, such as Figure 3 As shown, the optimal transfer problem aims to minimize the cost of transferring one distribution to another.

[0046] As a specific embodiment, the word unit representation and the substructure representation are regarded as two independent distributions. Among them C ij represents the transmission cost from the i-th word to the j-th substructure, represents the cosine distance from the i-th word to the j-th substructure. ij}, 1≤i≤N t , 1≤j≤N m , defined as the transmission scheme, the multi-word representation is obtained according to the following formula:

[0047]

[0048] D j ={i|a(i)=j,1≤i≤N t}

[0049]

[0050] in, represents the Frobenius dot product, a(i) represents the best alignment substructure index, and D j represents the index set, and the corresponding word is aligned with the jth substructure; Indicates D j The word-gram representations appearing in are averaged to obtain their fused multi-word-gram representations; ‖(·) represents the indicator function, Hp The multi-word representation representing the alignment of all substructures, N p is the number of multi-words.

[0051] S106, using contrastive learning loss function to optimize the alignment between word units and atoms, the alignment between multi-word units and substructures, and the alignment between sentences and molecules to obtain a trained text molecule retrieval model.

[0052] As an embodiment, three granularity contrastive learning loss functions are constructed to bring the two modalities closer through contrastive learning at the word-atom level, multi-word-substructure level, and sentence-molecule level, respectively. The contrastive learning loss function adopts a categorical cross entropy loss function to maximize the similarity scores between matching text-molecule pairs and minimize the scores between mismatching text-molecule pairs.

[0053] As a specific embodiment, the following training objectives are defined:

[0054]

[0055] in, and They represent word-to-sentence, multi-word-to-substructure, and sentence-to-molecule alignment losses, respectively. α and β represent coefficients for balancing the impact of different losses. These three losses are described in detail below:

[0056] (1) Token-atom alignment loss like Figure 4 As shown in Figure 1, the word-to-atom similarity matrix between word-to-word representation and atom representation is calculated; the similarity matrix is ​​then normalized by min-max normalization along the atom dimension; subsequently, the matrix is ​​further normalized along the word-to-word dimension to obtain alignment weights for unifying the dimensions of word-to-word and atom representations. After multiplying these alignment weights with the atom representation, the atom representation is updated by integrating the text information to Finally, sum pooling is applied to reduce the dimensionality of both representations and used to compute the final similarities at the token and atom levels.

[0057] As a specific embodiment, in a batch of samples of size B, the similarity between word-level and atomic-level samples is calculated. Next, the word-atom alignment loss It is defined as the average of two categorical cross entropy losses corresponding to the tasks of text molecules and molecule text retrieval, respectively:

[0058]

[0059] (2) Multi-word-substructure alignment loss In a similar way, we calculate the similarity between samples in the same batch at the multi-word and substructure levels. Therefore, the multi-word-substructure alignment loss The formula is as follows:

[0060]

[0061] (3) Sentence-molecule alignment loss Similar to other alignment levels, contrastive learning is used to calculate the CCE loss at the sentence-molecule level; specifically, the sentence-molecule alignment loss is calculated by maximizing the similarity between matched sentence-molecule pairs and minimizing the similarity between unmatched sentence-molecule pairs. The definition is as follows:

[0062]

[0063] in, It represents the similarity between the sentence representation of the qth sample and the molecule representation of the Kth sample.

[0064] In summary, according to the training method of the multi-granularity aligned text molecule retrieval model of the embodiment of the present invention, first, a molecular isomerism graph and a text description corresponding to each molecular isomerism graph are obtained, wherein the molecular isomerism graph includes atomic nodes, substructure nodes and molecular nodes, the molecular nodes are connected to all substructure nodes, and each substructure node is connected to the atomic nodes it includes; a text molecule retrieval model is constructed, wherein the text molecule retrieval model includes a text encoder and a molecular encoder; the text description corresponding to each molecular isomerism graph is input into the text encoder for encoding to obtain a word unit representation and a sentence representation; the molecular isomerism graph is input into the molecular encoder for encoding to obtain an atomic representation, a substructure representation and a molecular representation. The alignment relationship between word-gram representation and substructure representation is modeled as an optimal transfer problem, and the word-gram representations aligned with each substructure representation are aggregated to obtain multi-word-gram representations; the contrastive learning loss function is used to optimize the alignment between word-grams and atoms, the alignment between multi-word-grams and substructures, and the alignment between sentences and molecules to obtain a trained text molecule retrieval model; thus, the representations of atoms, substructures and molecules are obtained by constructing a hierarchical molecular heterogeneous graph, and the representations of multiple word-grams aligned with the same substructure are aggregated through the optimal transfer technology, and the text and molecular modalities are aligned at the word-gram-atom, multi-word-gram-substructure and sentence-molecule granularity through contrastive learning, thereby improving the accuracy of the retrieval results.

[0065] In order to implement the above embodiment, this embodiment also provides a multi-granularity aligned text molecule retrieval method.

[0066] Figure 5FIG. 1 is a flow chart of a multi-granularity aligned text molecule retrieval method according to an embodiment of the present invention. Figure 5 As shown, the method comprises the following steps:

[0067] S201, obtaining first data to be retrieved.

[0068] S202, input the first data to be retrieved into the trained text molecule retrieval model to weightedly calculate the similarity between the two modes of the first data to be retrieved and the second data at the word-atom level, the multi-word-substructure level and the sentence-molecule level, so as to obtain the second data with the highest similarity to the first data to be retrieved, so as to obtain the corresponding retrieval result, wherein the trained text molecule retrieval model is trained using the above-mentioned training method.

[0069] The multi-granularity aligned text molecule retrieval method of the disclosed embodiment, in the reasoning process of text molecule retrieval, weightedly calculates the similarity of two modalities at three levels and retrieves the molecules with the highest similarity; in the molecular text retrieval task, this process is performed in the opposite direction.

[0070] In order to implement the above-mentioned embodiment, the embodiment of the present disclosure provides a training device for a multi-granularity aligned text molecule retrieval model.

[0071] Figure 6 is a block diagram of a training device for a multi-granularity aligned text molecule retrieval model according to an exemplary embodiment. Figure 6 The device includes: a first acquisition module 1, a model construction module 2, a first encoding module 3, a second encoding module 4, an aggregation processing module 5, and an alignment optimization module 6.

[0072] Among them, the first acquisition module 1 is used to obtain the molecular isomerism graph and the text description corresponding to each molecular isomerism graph, wherein the molecular isomerism graph includes atomic nodes, substructure nodes and molecular nodes, the molecular nodes are connected to all substructure nodes, and each substructure node is connected to the atomic nodes it includes; the model construction module 2 is used to construct a text molecular retrieval model, wherein the text molecular retrieval model includes a text encoder and a molecular encoder; the first encoding module 3 is used to input the text description corresponding to each molecular isomerism graph into the text encoder for encoding to obtain word unit representation and sentence representation; the second encoding module 4 is used to input the molecular isomerism graph into the molecular encoder for encoding to obtain atomic representation, substructure representation and molecular representation; the aggregation processing module 5 is used to model the alignment relationship between word unit representation and substructure representation into an optimal transmission problem, and aggregate the word unit representation aligned with each substructure representation to obtain a multi-word unit representation; the alignment optimization module 6 is used to optimize the alignment between word units and atoms, the alignment between multi-word units and substructures, and the alignment between sentences and molecules using a contrastive learning loss function to obtain a trained text molecular retrieval model.

[0073] As an embodiment, the first encoding module 3 is further used to add a mark at the beginning of the text description corresponding to each molecular isomorphic graph to learn the sentence representation, and use a text encoder to encode the text description to obtain a word unit representation and a sentence representation.

[0074] As an embodiment, the aggregation processing module 5 is also used to treat the word-gram representation and the substructure representation as two independent distributions, and use the cosine distance as the transmission cost between the two representations; adopt the inexact neighboring point method of optimal transmission to obtain the optimal transmission plan with the minimum transmission cost; obtain the alignment relationship between the word-gram representation and the substructure representation according to the optimal transmission plan with the minimum transmission cost, so as to aggregate the word-gram representation aligned with each substructure representation to obtain a multi-word-gram representation.

[0075] As an embodiment, three granularity contrastive learning loss functions are constructed to bring the two modalities closer through contrastive learning at the word-atom level, multi-word-substructure level, and sentence-molecule level, respectively. The contrastive learning loss function adopts a categorical cross entropy loss function to maximize the similarity scores between matching text-molecule pairs and minimize the scores between mismatching text-molecule pairs.

[0076] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the training method for the multi-granularity aligned text molecule retrieval model, and will not be elaborated here.

[0077] In order to implement the above-mentioned embodiment, the embodiment of the present disclosure provides a multi-granularity aligned text molecule retrieval device.

[0078] Figure 7 is a block diagram of a multi-granularity aligned text molecule retrieval device according to an exemplary embodiment. Figure 7 The device includes: a second acquisition module 10 and a retrieval module 20.

[0079] Among them, the second acquisition module 10 is used to obtain the first data to be retrieved; the retrieval module 20 is used to input the first data to be retrieved into the trained text molecule retrieval model to weightedly calculate the similarity between the two modes of the first data to be retrieved and the second data at the word-atom level, the multi-word-substructure level and the sentence-molecule level, so as to obtain the second data with the highest similarity to the first data to be retrieved, and use the second data as the corresponding retrieval result, wherein the trained text molecule retrieval model is trained using the training method as described above.

[0080] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the multi-granularity aligned text molecule retrieval method, and will not be elaborated here.

[0081] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0082] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0083] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0084] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0085] It should be noted that in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0086] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0087] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0088] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0089] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0090] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0091] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0092] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A training method for a multi-granularity aligned text molecule retrieval model, characterized in that: The following steps are involved: Obtaining a molecular isomerism graph and a text description corresponding to each molecular isomerism graph, wherein the molecular isomerism graph includes atomic nodes, substructure nodes, and molecular nodes, the molecular nodes are connected to all substructure nodes, and each substructure node is connected to the atomic nodes included therein; Constructing a text molecule retrieval model, wherein the text molecule retrieval model includes a text encoder and a molecule encoder; Inputting the text description corresponding to each molecular isomorphic graph into the text encoder for encoding to obtain word unit representation and sentence representation; Inputting the molecular isomorphous graph into the molecular encoder for encoding to obtain an atomic representation, a substructure representation and a molecular representation; Modeling the alignment relationship between the word-gram representation and the substructure representation as an optimal transfer problem, and aggregating the word-gram representations aligned with each substructure representation to obtain a multi-word-gram representation; The contrastive learning loss function is used to optimize the alignment between word units and atoms, the alignment between multi-word units and substructures, and the alignment between sentences and molecules to obtain a trained text molecule retrieval model.

2. The training method of the multi-granularity aligned text molecule retrieval model according to claim 1, characterized in that: Inputting the text description corresponding to each molecular isomorphic graph into the text encoder for encoding to obtain word unit representation and sentence representation, including: A tag is added at the beginning of the text description corresponding to each molecular isomerization graph to learn sentence representation, and the text description is encoded using the text encoder to obtain word unit representation and sentence representation.

3. The training method of the multi-granularity aligned text molecule retrieval model as claimed in claim 1, characterized in that: Modeling the alignment relationship between the word-gram representation and the substructure representation as an optimal transfer problem, and aggregating the word-gram representations aligned with each substructure representation to obtain a multi-word-gram representation, including: Treat the word unit representation and the substructure representation as two independent distributions, and use the cosine distance as the transmission cost between the two representations; The optimal transmission scheme with the minimum transmission cost is obtained by using the inexact neighbor point method of optimal transmission; The alignment relationship between the word-gram representation and the substructure representation is obtained according to the optimal transmission scheme with the minimum transmission cost, so as to aggregate the word-gram representation aligned with each substructure representation to obtain a multi-word-gram representation.

4. The training method of the multi-granularity aligned text molecule retrieval model as claimed in claim 1, characterized in that: Contrastive learning loss functions of three granularities are constructed to bring the two modalities closer through contrastive learning at the word-atom level, multi-word-substructure level, and sentence-molecule level. Among them, the contrastive learning loss function adopts the categorical cross entropy loss function to maximize the similarity score between matching text-molecule pairs and minimize the score between mismatching text-molecule pairs.

5. A multi-granularity aligned text molecule retrieval method, characterized in that: The following steps are involved: Acquire first data to be retrieved; The first data to be retrieved is input into a trained text molecule retrieval model to weightedly calculate the similarity between the two modes of the first data to be retrieved and the second data at the word-atom level, the multi-word-substructure level and the sentence-molecule level to obtain the second data with the highest similarity to the first data to be retrieved to obtain the corresponding retrieval result, wherein the trained text molecule retrieval model is trained using the training method described in any one of claims 1 to 4.

6. A training device for a multi-granularity aligned text molecule retrieval model, characterized in that: include: A first acquisition module is used to acquire a molecular isomerization graph and a text description corresponding to each molecular isomerization graph, wherein the molecular isomerization graph includes atomic nodes, substructure nodes and molecular nodes, the molecular nodes are connected to all substructure nodes, and each substructure node is connected to the atomic nodes included therein; A model building module, used for building a text molecule retrieval model, wherein the text molecule retrieval model includes a text encoder and a molecule encoder; A first encoding module, used for inputting the text description corresponding to each molecular isomorphic graph into the text encoder for encoding to obtain word unit representation and sentence representation; A second encoding module, used for inputting the molecular isomorphous graph into the molecular encoder for encoding to obtain an atomic representation, a substructure representation and a molecular representation; An aggregation processing module, used for modeling the alignment relationship between the word-gram representation and the substructure representation as an optimal transmission problem, and aggregating the word-gram representation aligned with each substructure representation to obtain a multi-word-gram representation; The alignment optimization module is used to optimize the alignment between word units and atoms, the alignment between multi-word units and substructures, and the alignment between sentences and molecules by using contrastive learning loss function to obtain a trained text molecule retrieval model.

7. The training device for the multi-granularity aligned text molecule retrieval model according to claim 6, characterized in that: The first encoding module is further used for: A tag is added at the beginning of the text description corresponding to each molecular isomerization graph to learn sentence representation, and the text description is encoded using the text encoder to obtain word unit representation and sentence representation.

8. The training device for the multi-granularity aligned text molecule retrieval model according to claim 6, characterized in that: The aggregation processing module is also used to: Treat the word unit representation and the substructure representation as two independent distributions, and use the cosine distance as the transmission cost between the two representations; The optimal transmission scheme with the minimum transmission cost is obtained by using the inexact neighbor point method of optimal transmission; The alignment relationship between the word-gram representation and the substructure representation is obtained according to the optimal transmission scheme with the minimum transmission cost, so as to aggregate the word-gram representation aligned with each substructure representation to obtain a multi-word-gram representation.

9. The training device for the multi-granularity aligned text molecule retrieval model according to claim 6, characterized in that: Contrastive learning loss functions of three granularities are constructed to bring the two modalities closer through contrastive learning at the word-atom level, multi-word-substructure level, and sentence-molecule level. Among them, the contrastive learning loss function adopts the categorical cross entropy loss function to maximize the similarity score between matching text-molecule pairs and minimize the score between mismatching text-molecule pairs.

10. A multi-granularity aligned text molecule retrieval device, characterized in that: include: A second acquisition module, used to acquire first data to be retrieved; A retrieval module is used to input the first data to be retrieved into a trained text molecule retrieval model to weightedly calculate the similarity between the two modes of the first data to be retrieved and the second data at the word-atom level, the multi-word-substructure level and the sentence-molecule level to obtain the second data with the highest similarity to the first data to be retrieved, and use the second data as the corresponding retrieval result, wherein the trained text molecule retrieval model is trained using the training method described in any one of claims 1 to 4.

Images