Training method of molecular data processing model, molecular data processing method, device, equipment, storage medium and program product

Abstract

The invention provides a molecular data processing model training method and device, a molecular data processing method and device, equipment, a storage medium and a program product. The method comprises the following steps: iteratively executing a plurality of training batches on a molecular marker, and executing the following processing in each training batch: performing feature interaction processing on a molecular description text, molecular graph data and a first query vector sequence of the current training batch through the molecular marker to obtain an interaction query sequence, querying a codebook of the current training batch based on the interactive query sequence to obtain a molecular marker sequence, determining a first loss value based on the interactive query sequence and the molecular marker sequence, and updating the molecular marker, the first query vector sequence and the codebook based on the first loss value; and based on the molecular markers after multiple training batches, executing a training task of the molecular data processing model. According to the method and the device, the consistency of multi-modal data representation can be improved, and the generalization ability of a molecular data processing model is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a training method for a molecular data processing model, a molecular data processing method, a device, an equipment, a storage medium and a program product. Background Art

[0002] The molecular data processing method in the related technology integrates molecular information by fusing molecular data of different modalities (such as molecular graph data and text data), such as using a multi-head attention mechanism or a cross-attention mechanism. Although this method can process multimodal data of molecules and texts, since data of different modalities have different structures and characteristics, the processing of multimodal data still has limitations, which affects the performance of the molecular data processing model. Summary of the invention

[0003] The embodiments of the present application provide a training method, a molecular data processing method, an apparatus, a device, a storage medium, and a program product for a molecular data processing model, which can improve the consistency of multimodal data representation and improve the generalization ability of the molecular data processing model.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application embodiment provides a method for training a molecular data processing model, wherein the molecular data processing model is trained with the assistance of a molecular marker, and the method comprises:

[0006] A plurality of training batches are iteratively performed on the molecular marker, wherein the following processes are performed in each training batch:

[0007] The molecular marker performs feature interactive processing on the molecular description text, the molecular graph data and the first query vector sequence of the current training batch to obtain an interactive query sequence,

[0008] Based on the interactive query sequence, the codebook of the current training batch is queried to obtain a molecular marker sequence,

[0009] Determining a first loss value based on the interactive query sequence and the molecular marker sequence,

[0010] Update the molecular marker, the first query vector sequence, and the codebook based on the first loss value, wherein the updated molecular marker, the first query vector sequence, and the codebook are used for a next training batch of the molecular marker;

[0011] Based on the molecular markers after the multiple training batches, the training task of the molecular data processing model is performed.

[0012] The present application provides a molecular data processing method, the method comprising:

[0013] Obtaining molecular data to be processed and prompts to be processed;

[0014] In response to the molecular data to be processed representing a molecular structure, performing molecular marker generation processing on the molecular data to be processed by a molecular marker to obtain a molecular marker sequence of the molecular data to be processed;

[0015] Through the molecular data processing model, molecular description text prediction processing is performed based on the molecular marker sequence of the molecular data to be processed and the embedded representation of the prompt to be processed, so as to obtain the molecular description text of the molecular data to be processed, wherein the molecular marker and the molecular data processing model are trained by the training method of the molecular data processing model provided in the embodiment of the present application.

[0016] The present application provides a molecular data processing method, the method comprising:

[0017] Obtaining molecular data to be processed and prompts to be processed;

[0018] In response to the fact that the molecular data to be processed is text data, molecular marker prediction processing is performed through the molecular data processing model based on the embedded representation of the molecular data to be processed and the embedded representation of the prompt to be processed, so as to obtain a molecular marker sequence of the molecular data to be processed, wherein the molecular marker and the molecular data processing model are trained by the training method of the molecular data processing model provided in an embodiment of the present application.

[0019] The present application embodiment provides a training device for a molecular data processing model, comprising:

[0020] A first training module is used to iteratively execute multiple training batches for the molecular marker, wherein the following processing is performed in each training batch: feature interaction processing is performed on the molecular description text, the molecular graph data and the first query vector sequence of the current training batch by the molecular marker to obtain an interactive query sequence, querying the codebook of the current training batch based on the interactive query sequence to obtain a molecular marker sequence, determining a first loss value based on the interactive query sequence and the molecular marker sequence, and updating the molecular marker, the first query vector sequence and the codebook based on the first loss value, wherein the updated molecular marker, the first query vector sequence and the codebook are used for the next training batch of the molecular marker;

[0021] The second training module is used to perform the training task of the molecular data processing model based on the molecular markers after the multiple training batches.

[0022] The present application embodiment provides a molecular data processing device, comprising:

[0023] A data processing module, used to obtain molecular data to be processed and prompts to be processed;

[0024] The data processing module is further used for performing molecular marker generation processing on the molecular data to be processed by a molecular marker to obtain a molecular marker sequence of the molecular data to be processed in response to the molecular data to be processed representing a molecular structure;

[0025] A prediction module is used to perform molecular description text prediction processing based on the molecular marker sequence of the molecular data to be processed and the embedded representation of the prompt to be processed through a molecular data processing model to obtain the molecular description text of the molecular data to be processed, wherein the molecular marker and the molecular data processing model are trained by the training method of the molecular data processing model provided in the embodiment of the present application.

[0026] The present application embodiment provides a molecular data processing device, comprising:

[0027] A data processing module, used to obtain molecular data to be processed and prompts to be processed;

[0028] A prediction module is used for, in response to the molecular data to be processed being text data, performing molecular marker prediction processing based on the embedded representation of the molecular data to be processed and the embedded representation of the prompt to be processed through the molecular data processing model to obtain a molecular marker sequence of the molecular data to be processed, wherein the molecular marker and the molecular data processing model are trained by the training method of the molecular data processing model provided in an embodiment of the present application.

[0029] An embodiment of the present application provides an electronic device, the electronic device comprising:

[0030] A memory for storing computer executable instructions or computer programs;

[0031] The processor is used to implement the training method of the molecular data processing model provided in the embodiment of the present application, or implement the molecular data processing method provided in the embodiment of the present application when executing the computer executable instructions or computer programs stored in the memory.

[0032] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which are used to implement the training method of the molecular data processing model provided in the embodiment of the present application, or implement the molecular data processing method provided in the embodiment of the present application when executed by a processor.

[0033] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, the training method of the molecular data processing model provided in the embodiment of the present application is implemented, or the molecular data processing method provided in the embodiment of the present application is implemented.

[0034] The embodiments of the present application have the following beneficial effects:

[0035] By executing multiple training batches on the molecular marker, performing feature interactive processing on the molecular graph data and the molecular description text, continuously updating the first query vector sequence and the code book, and using the interactive query sequence obtained by the feature interactive processing to map the molecular graph data to a molecular marker sequence, the molecular representation sequence and the molecular description text representation are aligned, thereby improving the consistency of multimodal data representation. This unified representation eliminates the encoding differences between different modalities, so that the trained molecular data processing model can more naturally process and fuse multimodal information, accurately capture and align the deep relationship between molecules and text. Specifically, by performing feature interactive processing on the molecular graph data and the molecular description text, the model can learn the correlation between the molecular graph data and the molecular description text. This interaction can reveal the implicit relationship between the molecular structure and its description text, thereby improving the molecular data processing model's understanding of molecular information and achieving the beneficial effect of improving the generalization ability of the molecular data processing model. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a structural diagram of the molecular data processing system architecture provided in an embodiment of the present application;

[0037] Figure 2A is a first structural diagram of a server provided in an embodiment of the present application;

[0038] Figure 2B is a second structural diagram of a server provided in an embodiment of the present application;

[0039] Figure 2C is a third structural diagram of a server provided in an embodiment of the present application;

[0040] Figure 3A It is a first flow chart of the training method of the molecular data processing model provided in the embodiment of the present application;

[0041] Figure 3B It is a second flow chart of the training method of the molecular data processing model provided in the embodiment of the present application;

[0042] Figure 3C is a third flow chart of the training method of the molecular data processing model provided in the embodiment of the present application;

[0043] Figure 3D 4 is a schematic diagram of a fourth flow chart of a method for training a molecular data processing model provided in an embodiment of the present application;

[0044] Figure 3E is a fifth flow chart of the training method of the molecular data processing model provided in the embodiment of the present application;

[0045] Figure 3F 6 is a schematic diagram of a training method for a molecular data processing model provided in an embodiment of the present application;

[0046] Figure 3G 7 is a schematic diagram of a seventh flow chart of a method for training a molecular data processing model provided in an embodiment of the present application;

[0047] Figure 3H 8 is a schematic diagram of an eighth flow chart of a method for training a molecular data processing model provided in an embodiment of the present application;

[0048] Fig. 3I is a ninth flow chart of the method for training a molecular data processing model provided in an embodiment of the present application;

[0049] Figure 3J 10 is a schematic diagram of a training method for a molecular data processing model provided in an embodiment of the present application;

[0050] Figure 3K This is an eleventh flow chart of the training method of the molecular data processing model provided in the embodiment of the present application;

[0051] Figure 3L It is a twelfth flow chart of the training method of the molecular data processing model provided in the embodiment of the present application;

[0052] Figure 3M is a thirteenth flow chart of the training method of the molecular data processing model provided in the embodiment of the present application;

[0053] Figure 4 It is a first flow chart of the molecular data processing method provided in an embodiment of the present application;

[0054] Figure 5 is a second flow chart of the molecular data processing method provided in an embodiment of the present application;

[0055] Figure 6 It is a flowchart of a method for processing molecular data in a molecular property prediction scenario provided in an embodiment of the present application;

[0056] Fig. 7A is a first schematic diagram of an optional structure of a molecular data processing model provided in an embodiment of the present application;

[0057] Figure 7Bis a second schematic diagram of an optional structure of a molecular data processing model provided in an embodiment of the present application;

[0058] Figure 7C is a schematic diagram of the training principle of the first language understanding model provided in the embodiment of the present application;

[0059] Fig.7D It is a schematic diagram of the training principle of the second language comprehension model adopted in the embodiment of the present application.

[0060] It should be pointed out that the above-mentioned "first" and "second" are only used to distinguish different solutions, and do not represent the degree of superiority or inferiority of the solutions or the priority in the implementation process. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0062] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0063] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0064] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0065] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0066] The relevant data collection and processing in the embodiments of this application should be strictly in accordance with the requirements of relevant laws and regulations when applied in examples, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0067] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0068] 1) In response, it is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or have a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed are executed.

[0069] 2) Molecular graph data is a graphical data format used to represent and store molecular structures. In this format, molecules are represented as a graph, where nodes represent atoms and edges represent chemical bonds. Molecular graph data can capture the topological structure and chemical properties of molecules and is a commonly used data form in the fields of chemical informatics and molecular modeling. The following are some key components of molecular graph data:

[0070] Node (atom): In a molecular graph, each node represents an atom. The node usually contains the type of atom (such as carbon, hydrogen, oxygen, etc.), as well as the properties of the atom (such as charge, whether it is part of a ring, etc.).

[0071] Edges (Bonds): Edges between nodes represent chemical bonds between atoms. Edges can be directed (representing single, double, or triple bonds) or undirected (representing any type of bond). Edges may also contain properties of the bond, such as the strength or type of the bond (single, double, triple, or aromatic).

[0072] Graph structure: The structure of a molecular graph is defined by a collection of nodes and edges. It can be simple, a small molecule graph containing only a few atoms and bonds, or complex, a large molecule graph containing hundreds of atoms and bonds.

[0073] Additional information: In addition to the basic graph structure, molecular graph data may also include other information, such as the three-dimensional coordinates of the molecule, molecular weight, molecular formula, functional group information, etc.

[0074] 3) Molecular representation data, also known as Simplified Molecular Input Line Entry System (SMILES) and SMILES strings, is a simplified linear text representation for describing the structure of chemical substances. SMILES strings are widely used in the fields of cheminformatics and drug design.

[0075] A SMILES string (molecule representation data) consists of the following basic elements:

[0076] Atoms: Atoms are represented using symbols from the periodic table, such as "C" for carbon and "H" for hydrogen.

[0077] Chemical bonds: Use specific symbols to represent how atoms are connected. Single bonds do not require special symbols, double bonds are represented by an equal sign "=", and triple bonds are represented by a pound sign "#".

[0078] Branching: Use parentheses to indicate branching in a molecule.

[0079] Ring structure: Use numbers to mark the opening and closing positions of the ring structure.

[0080] Charge: Use the plus "+" and minus "-" signs to indicate the charge of an atom.

[0081] Here are some examples of SMILES strings:

[0082] Water: O or H2O.

[0083] Ethanol: CO or COC.

[0084] Benzene: c1cccccc1.

[0085] Acetone: CC(=O)C.

[0086] In the above SMILES string c1ccCcc1 for benzene, c represents a carbon atom and the number 1 indicates the closure of the ring, i.e. the first and last carbon atoms form a ring.

[0087] The advantage of SMILES strings is their simplicity and readability, which allows chemists to quickly record and share molecular structures. At the same time, SMILES strings can also be parsed by computer programs to generate two-dimensional and three-dimensional structures of molecules, as well as to calculate and analyze various chemical information.

[0088] 4) Molecular description text is a text form that uses natural language or semi-structured language to describe the structure of chemical molecules, including the composition of the molecule, the type of atoms, the type of chemical bonds, the geometric structure of the molecule, and possible other chemical properties.

[0089] The following are some key elements of a molecule description text:

[0090] Atomic information: describes the atoms present in a molecule, including the type of atoms (such as carbon, hydrogen, oxygen, etc.) and the number of atoms.

[0091] Chemical bond information: describes how atoms are connected, including single bonds, double bonds, triple bonds, etc.

[0092] Molecular structure: describes the spatial structure of a molecule and may include the stereochemical configuration of the molecule.

[0093] Functional group: indicates the specific functional group in the molecule, such as hydroxyl, keto, amino, etc.

[0094] Molecular properties: may include physical and chemical properties of the molecule, such as molecular weight, boiling point, melting point, etc.

[0095] Chemical reaction characteristics: describe the effects and changes of molecules in chemical reactions.

[0096] For example, the molecular description text for ethanol can be expressed as: Ethanol (EthylAlcohol) is commonly known as alcohol and liquor. It is a kind of alcohol compound with a molecular formula of C2H6O. It consists of two carbon atoms connected by a single bond. The first carbon atom is connected to three hydrogen atoms, and the second carbon atom is connected to two hydrogen atoms and a hydroxyl group (-OH). Ethanol has good flammability and is a commonly used fuel, solvent, disinfectant, etc. It is widely used in organic synthesis.

[0097] 5) Transformer refers to a time series model based on the self-attention mechanism. The encoder part can effectively encode time series information. Its processing ability of time series information is much better than that of the long short-term memory network (LSTM), and its speed is fast. It is widely used in natural language processing, computer vision, machine translation, speech recognition and other fields.

[0098] 6) Large Language Model (LLM) is a large-scale language model designed to understand and generate human language. They are trained on large amounts of text data and can perform a wide range of tasks, including text summarization, translation, sentiment analysis, etc. Large language models are characterized by their large scale and billions of parameters that help them learn complex patterns in language data. They are usually based on deep learning architectures. Large language models refer to deep learning models trained with large amounts of text data, containing billions or even larger parameters. They can be used to generate natural language text and understand the meaning of natural language text. Through training, the model can learn the statistical laws and semantic associations of language to build a huge language knowledge base, thereby simulating human language understanding and generation capabilities. Large language models have the following characteristics:

[0099] Learning ability: Through training on massive amounts of text data, large language models can learn a wealth of language knowledge and expressions, including grammar, semantics, and common expression habits.

[0100] Pattern recognition: Large language models can recognize common text patterns and semantic associations, such as the co-occurrence relationship between words, the logical structure and semantic roles of sentences, etc.

[0101] Contextual understanding: Large language models can capture contextual information in text, understand the impact of previous text on subsequent text, and generate appropriate responses based on the context.

[0102] Generation capability: The large language model can generate relevant natural language text based on the input information, including answering questions, generating articles, and conducting conversations.

[0103] Resolving ambiguity: Despite the polysemy and ambiguity of language, large language models resolve ambiguity through contextual information and language regularities, providing more accurate and appropriate text generation or understanding.

[0104] The application scenarios of large language models are very broad. They can be applied to intelligent customer service, intelligent question and answer, natural language generation, advertising recommendation, games and other fields. They can improve the efficiency and accuracy of human-computer interaction and enhance user experience.

[0105] 7) Querying Transformer (Q-Former) is a query-based transformer model that uses a learnable query vector to interact with the molecular features extracted by the frozen encoder. These queries are critical for extracting relevant information from the molecular features. Q-Former includes a Molecule Transformer and a Text Transformer, which share a self-attention layer. The Molecule Transformer adds a cross-attention layer between the self-attention and feed-forward layers, and the architecture of the text transformer can be built based on the Deep Bidirectional Transformer for Language Understanding (BERT). Q-Former adopts a cross-attention mechanism in which the query vector selectively focuses on different aspects of the molecular features, enabling the model to capture the key details necessary to understand the properties of the molecule and generate text descriptions.

[0106] 8) Molecular Tokenizer is a model used to segment molecular graph data into multiple smaller, meaningful units or "tags". When processing molecular data, these molecular tags can help identify and predict the structure and function of molecules. Specifically, a molecular tokenizer can be seen as a model that converts molecular graph data into a format that can be processed by a machine learning model.

[0107] 9) The molecular data processing model is a large language model based on the Transformer architecture, which is specifically used to process molecular data. It can learn the complex distribution of molecular data through pre-training and can be used for molecular property prediction, drug design and other molecule-related tasks.

[0108] 10) Graph neural network (GNN) is a neural network that takes as input graph data containing points, edges and their attributes and performs predictions.

[0109] The molecular data processing methods in related technologies use independent molecular embedding and text embedding, and then integrate this information through some form of fusion mechanism, such as multi-head attention mechanism or cross-attention mechanism, by calculating the similarity between different modalities and mapping them to a shared representation space. Although this method can process multimodal data of molecules and text, it still has limitations in the consistency of data representation and the complexity of generation tasks.

[0110] The technical solutions of the relevant technologies mainly include the following aspects:

[0111] 1) Multimodal alignment models: These models obtain embedding representations of different modalities by encoding molecules and text independently, and then use alignment mechanisms to achieve cross-modal interactions.

[0112] 2) Self-supervised learning methods: Use self-supervised learning techniques such as contrastive learning to learn general multimodal representations by designing specific pre-training tasks, such as molecular-text contrastive learning training tasks.

[0113] 3) Molecular generation models: Traditional molecular generation models usually use graph neural networks (GNNs) or generation models based on SMILES representation to generate new molecular structures, but these models usually only focus on the molecules themselves and do not involve multimodal generation.

[0114] Multimodal alignment models refer to a class of models that aim to map data from different modalities into a shared representation space. Such models usually include separate encoders for processing data from different modalities such as text and molecules. For text data, commonly used encoders are pre-trained language models that are trained on a large amount of natural language text and can generate rich text embedding representations. For molecular data, models usually use graph neural networks (GNNs) or sequence models based on SMILES representation to generate molecular embeddings. GNNs use the structure of graphs to encode the atoms and bonds of molecules as nodes and edges, while SMILES-based models represent molecules as character sequences and generate corresponding embeddings. In multimodal alignment models, after being encoded by their respective encoders, data from different modalities will go through some form of alignment mechanism to achieve cross-modal interaction. This alignment mechanism usually uses a multi-head attention mechanism or a cross-attention mechanism to map different modalities into a shared representation space by calculating the similarity between them. In this shared space, the embeddings of different modalities can be used for various cross-modal tasks, such as cross-modal retrieval, cross-modal generation, etc.

[0115] Self-supervised learning methods are a type of method that uses unlabeled data to train models, and are particularly suitable for multimodal tasks. Unlike supervised learning, which relies on a large amount of labeled data, self-supervised learning enables the model to automatically learn effective feature representations from raw data by designing specific pre-training tasks. In multimodal learning, self-supervised learning methods usually design some synthetic tasks or signals so that the model can autonomously obtain supervisory signals from the input data. These tasks can include contrastive learning, masked language modeling, generative adversarial networks, and many other forms. Among them, contrastive learning is a common self-supervised learning method that trains the model by comparing positive and negative sample pairs to learn universal multimodal representations. Specifically, contrastive learning generates paired samples from the dataset, and the two data instances in a pair of samples either belong to the same category or to different categories. In the molecular text contrastive learning training task, the positive sample pair can be a molecule and its corresponding text description, while the negative sample pair is a combination of a molecule and an unrelated text. During the learning process, the model will try to make the representation of the positive sample pair as close as possible and the representation of the negative sample pair as far away as possible. This approach forces the model to learn which features are important factors in distinguishing the similarities between modalities, thereby gradually building effective representations without explicit labels.

[0116] Molecular generative models are a class of machine learning models that focus on generating and optimizing chemical molecular structures. They use data-driven methods to learn to generate molecules that meet specific physical and chemical properties from a combination of atoms and bonds. Molecular generative models have important applications in drug discovery, material design, and chemical synthesis. Traditional molecular generative models usually rely on graph neural networks (GNNs) or sequence models based on SMILES representation for modeling. GNN models learn the global representation of graphs by recursively passing and aggregating information between nodes (atoms) and edges (chemical bonds) of molecular graphs. This process is similar to diffusing information in a molecular graph, so that the final representation of each node contains the structural information of the node and its neighborhood. This method can capture the complex topological features of molecular structures and is therefore very suitable for generating new molecules. By learning the distribution of molecular graphs on training data, GNNs can construct new molecular structures that meet specific requirements during the generation phase. In addition, GNN models can also integrate attribute information of nodes and edges, such as the type of atoms and the type of bonds, to more accurately reflect the chemical properties of molecules. On the other hand, sequence models based on SMILES representation use a simplified molecular linear representation to convert molecules into string sequences. This representation flattens the two-dimensional structure of a molecule into a one-dimensional sequence, retaining the composition and connectivity information of the molecule. Using a sequence generation model, such as a recurrent neural network (RNN) or Transformer, the model can predict the next character during the character-by-character generation process to construct a complete SMILES sequence. The advantage of this method lies in the simplicity of its generation process and its easy integration with language models. By training on a large amount of SMILES data, the model can capture the grammatical rules and common structural patterns of chemical molecules, thereby generating or synthesizing reasonable and novel molecules.

[0117] The limitations of related technologies are mainly reflected in the following aspects:

[0118] 1) Insufficient alignment and fusion of multimodal data: Existing multimodal alignment models usually use independent encoders and alignment mechanisms to process data of different modalities, which may lead to information loss and insufficient alignment, especially when dealing with data of different complexities and scales. The representations produced by independent encoders may not fully capture the deep relationship between different modalities, which limits the performance of the model in multimodal tasks.

[0119] 2) Complexity of self-supervised learning task design: Self-supervised learning methods rely on carefully designed pre-training tasks, and the selection and design of these tasks require a lot of experiments and experience. Improper task design may result in the representation learned by the model not having good generalization ability. In addition, the training of self-supervised learning models usually requires a lot of computing resources, especially when dealing with large-scale multimodal data.

[0120] 3) Limitations of molecular generation models: Traditional molecular generation models, such as GNN and SMILES-based sequence models, although they perform well in generating molecules, have limitations in dealing with interactions with other modalities such as text. These models usually only focus on the structural generation of molecules, while ignoring the fusion of multimodal information, making it difficult to achieve multimodal joint generation and understanding tasks. In addition, these models may have difficulty in generating molecules with specific properties, especially when faced with complex design requirements.

[0121] The embodiments of the present application provide a training method for a molecular data processing model, a molecular data processing method, an apparatus, a device, a computer-readable storage medium, and a computer program product, which can improve the consistency of multimodal data representation and improve the generalization ability of the molecular data processing model. The following describes an exemplary application of the electronic device provided by the embodiments of the present application. The electronic device provided by the embodiments of the present application can be implemented as various types of terminals such as laptops, tablet computers, desktop computers, set-top boxes, smart phones, smart speakers, smart watches, smart TVs, and vehicle-mounted terminals, and can also be implemented as servers.

[0122] See also Figure 1 , Figure 1 is a schematic diagram of the structure of the molecular data processing system architecture provided in the embodiment of the present application, Figure 1 The server 100, the terminal device 200 and the network 300 are involved. The terminal device 200 is connected to the server 100 via the network 300, wherein the network 300 can be a wide area network or a local area network, or a combination of the two.

[0123] In some embodiments, the embodiments of the present application can be implemented collaboratively by a server and a terminal device. For example, the server 100 obtains a molecular data processing model through the training method of the molecular data processing model provided in the embodiments of the present application, and the terminal device 200 sends the molecular data to be processed and the prompt to be processed to the server 100. The server 100 obtains the molecular description text of the molecular data to be processed or the molecular label sequence of the molecular data to be processed through the molecular data processing method provided in the embodiments of the present application, and sends the molecular description text of the molecular data to be processed or the molecular label sequence of the molecular data to be processed (which can be used to reconstruct the molecular structure) as the processing result to the terminal device 200.

[0124] Here, the server 100 can be a single server. In this case, the training method and the molecular data processing method of the molecular data processing model provided by the embodiment of the present application can be implemented by the same server. The server 100 can also be a cluster of servers. In the case where the server 100 is a server cluster, the training method and the molecular data processing method of the molecular data processing model provided by the embodiment of the present application can be implemented by different servers, which is not limited by the embodiment of the present application.

[0125] In other embodiments, the embodiments of the present application can be implemented by a terminal device alone. The terminal device 200 sends a request to the server 100, and the server 100 receives the request and sends a molecular data processing model for performing the molecular data processing method provided in the embodiments of the present application to the terminal device 200. The terminal device 200 receives the molecular data processing model sent by the server and downloads it locally, and obtains the molecular description text of the molecular data to be processed or the molecular label sequence of the molecular data to be processed through the molecular data processing model.

[0126] Take the server for training molecular data processing models as an example, see Figure 2A , Figure 2A is a first structural diagram of a server provided in an embodiment of the present application, Figure 2A The server 100-1 shown includes: at least one processor 110-1, a memory 130-1 and at least one network interface 120-1. The various components in the server 100-1 are coupled together via a bus system 140-1. It is understood that the bus system 140-1 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 140-1 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 140-1 is not described in detail. Figure 2A Various buses are labeled as bus system 140 - 1 .

[0127] Processor 110-1 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0128] Memory 130-1 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. Memory 130-1 may optionally include one or more storage devices that are physically remote from processor 110-1.

[0129] The memory 130-1 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 130-1 described in the embodiment of the present application is intended to include any suitable type of memory.

[0130] In some embodiments, the memory 130 - 1 is capable of storing data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0131] Operating system 131-1, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0132] A network communication module 132-1, used to reach other electronic devices via one or more (wired or wireless) network interfaces 120-1, exemplary network interfaces 120-1 include: Bluetooth, Wireless LAN (WiFi), and Universal Serial Bus (USB), etc.;

[0133] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2A The training device 133 of the molecular data processing model stored in the memory 130-1 is shown, which can be software in the form of a program and a plug-in, including the following software modules: a first training module 1331 and a second training module 1332. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.

[0134] For example, the server is used for molecular data processing, see Figure 2B , Figure 2B is a second structural diagram of a server provided in an embodiment of the present application, Figure 2B The server 100-2 shown includes: at least one processor 110-2, a memory 130-2 and at least one network interface 120-2. The various components in the server 100-2 are coupled together via a bus system 140-2. It is understood that the bus system 140-2 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 140-2 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 140-2 is not described in detail. Figure 2BVarious buses are labeled as bus system 140-2. For the detailed description of processor 110-2 and memory 130-2, please refer to the above, which will not be repeated here.

[0135] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2B The molecular data processing device 134 stored in the memory 130-2 is shown, which can be software in the form of a program and a plug-in, etc., including the following software modules: a data processing module 1341 and a prediction module 1342. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.

[0136] For example, the server is used for molecular data processing, see Figure 2C , Figure 2C is a third structural diagram of a server provided in an embodiment of the present application, Figure 2C The server 100-3 shown includes: at least one processor 110-3, a memory 130-3 and at least one network interface 120-3. The various components in the server 100-3 are coupled together through a bus system 140-3. It is understood that the bus system 140-3 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 140-3 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 140-3 is not described in detail. Figure 2C In the figure, various buses are labeled as bus system 140-3. For the detailed description of processor 110-3 and memory 130-3, please refer to the above text, which will not be repeated here.

[0137] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2C The molecular data processing device 135 stored in the memory 130-3 is shown, which can be software in the form of a program and a plug-in, including the following software modules: a data processing module 1351 and a prediction module 1352. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.

[0138] In some embodiments, the terminal device or server can realize the training method or molecular data processing method of the molecular data processing model provided by the embodiment of the present application by running various computer executable instructions or computer programs. For example, computer executable instructions can be commands, machine instructions or software instructions at the microprogram level. The computer program can be a native program or software module in the operating system; it can be a local (Native) application (APPlication, APP), that is, a program that needs to be installed in the operating system to run, such as a chemical database APP or a scientific research collaboration APP, etc.; it can also be a small program that can be embedded in any APP, that is, a program that can be run only by downloading it to a browser environment. In short, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module or plug-in in any form.

[0139] In other embodiments, the device provided in the embodiments of the present application can be implemented in hardware. As an example, the device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the training method or molecular data processing method of the molecular data processing model provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (Application Specific Integrated Circuit, ASIC), digital signal processors (Digital Signal Processor, DSP), programmable logic devices (Programmable Logic Device, PLD), complex programmable logic devices (Complex Programmable Logic Device, CPLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other electronic components.

[0140] The following will combine the exemplary application and implementation of the server provided in the embodiment of the present application, taking the server as the execution subject, to illustrate the training method of the molecular data processing model provided in the embodiment of the present application. Figure 3A , Figure 3A is a first flow chart of the training method of the molecular data processing model provided in the embodiment of the present application, which will be combined with Figure 3A The steps shown are explained.

[0141] In step 101 , multiple training batches are iteratively executed on the molecular marker, wherein the processes of the following steps 1011 to 1014 are executed in each training batch.

[0142] See also Fig. 7A , Fig. 7Ais a first schematic diagram of an optional structure of a molecular data processing model provided in an embodiment of the present application, Fig. 7A An optional structure of a molecular marker is shown, in which molecular graph data is input into a pre-trained molecular encoder to obtain encoding features, and the encoding features, a learnable first query vector sequence and a molecular description text are input into a query converter for feature interaction processing (corresponding to step 1011 below) to obtain an interactive query sequence, and a molecular marker sequence is obtained by querying the neighbors of the interactive query sequence in the learnable encoding book.

[0143] In step 1011, a molecular marker is used to perform feature interaction processing on the molecular description text, the molecular graph data and the first query vector sequence of the current training batch to obtain an interactive query sequence.

[0144] In some embodiments, see Figure 3B , Figure 3A The step 1011 shown can be implemented by following the steps 10111 to 10113, which are described in detail below.

[0145] In step 10111, a first attention processing is performed on the text features of the molecule description text and the first query vector sequence to obtain a first attention feature.

[0146] In some embodiments, see Figure 3C , Figure 3B The step 10111 shown can be implemented by following the steps 201 to 204, which are described in detail below.

[0147] In step 201, a first attention score matrix is ​​determined based on text features of the molecule description text and a first query vector sequence.

[0148] In some embodiments, the text features of the molecular description text can be obtained by: performing word segmentation processing on the molecular description text to obtain input units; performing token embedding coding (Token Embeddings) processing on the input units to obtain token embedding features; performing self-attention coding processing on the token embedding features to obtain text features of the molecular description text.

[0149] For example, punctuation marks such as spaces, periods, and commas can be used as segmentation marks to segment the molecular description text, that is, to divide the molecular description text into input units. Third-party segmentation tools can also be used to segment the molecular description text, such as Jieba, NLTK, SpaCy, etc. These tools can achieve accurate Chinese and English segmentation through algorithms and language models.

[0150] For example, a word embedding model (such as Word2Vec, GloVe, etc.) can be used to perform word embedding encoding processing on the input unit, each input unit is represented as a vector of a fixed length, and word embedding features (sequences) are formed in order.

[0151] For example, the word embedding features can be used as a query vector (Query, Q), a key vector (Key, K) and a value vector (Value, V). The attention score is obtained by calculating the dot product of Q and K. Next, the attention score is normalized, for example, a normalization function (such as a softmax function) is applied to convert the attention score into a probability distribution as an attention weight. The attention weight is used to weight the word embedding features (corresponding to V above) to generate a new feature representation. The new feature representation can be linearly transformed through a linear layer to obtain a rich representation that includes the correlation between different positions in the word embedding features, that is, the text features of the molecular description text.

[0152] In some embodiments, determining the first attention score matrix based on the text features of the molecular description text and the first query vector sequence can be achieved in the following way: using the first query vector sequence as the query vector (Query, Q), using the text features as both the key vector (Key, K) and the value vector (Value, V), and obtaining the first attention score matrix by calculating the dot product of Q and K.

[0153] For example, assuming that Q and K contain three elements (q1, q2, q3) and (k1, k2, k3) respectively, the obtained first attention score matrix can be expressed as: {(q1k1: 0.1, q1k2: 0.2, q1k3: 0.7), (q2k1: 0.4, q2k2: 0.3, q2k3: 0.3), (q3k1: 0.2, q3k2: 0.5, q3k3: 0.3)}.

[0154] In step 202, a mask matrix of the first attention score matrix is ​​obtained.

[0155] In some embodiments, a causal mask may be used as a mask matrix for the first attention score matrix.

[0156] For example, the mask matrix can be expressed as: {(1, -∞, -∞), (1, 1, -∞), (1, 1, 1)}.

[0157] In step 203, the first attention score matrix is ​​masked based on the mask matrix, and the obtained mask score matrix is ​​normalized to obtain a first attention weight matrix.

[0158] In some embodiments, the mask matrix is ​​multiplied by the corresponding position elements of the first attention score matrix to obtain a mask score matrix.

[0159] Continuing with the above example, the mask score matrix can be expressed as: {(q1k1: 0.1, q1k2: -∞, qlk3: -∞), (q2k1: 0.4, q2k2: 0.3, q2k3: -∞), (q3k1: 0.2, q3k2: 0.5, q3k3: 0.3)}.

[0160] Continuing with the above example, the mask score matrix can be normalized by the softmax function, and the obtained first attention weight matrix can be expressed as: {(qlk1: 1.00, q1k2: 0.00, qlk3: 0.00), (q2k1: 0.57, q2k2: 0.43, q2k3: 0.00), (q3k1: 0.31, q3k2: 0.41, q3k3: 0.28)}.

[0161] In step 204, the text features are weighted and summed using a first attention weight matrix to obtain a first attention feature.

[0162] Continuing with the above example, the text features (v1, v2, v3) are weighted and summed through the first attention weight matrix. For q1: qlk1*v1=v1; for q2: q2k1*v1+q2k2*v2=0.57vl+0.43v2; for q3: q3kl*v1+q3k2*v2+q3k3*v3=0.31v1+0.41v2+0.28v3. The first attention feature can be expressed as: (v1, 0.57vl+0.43v2, 0.31v1+0.41v2+0.28v3). It can be seen that q1 only pays attention to k1, q2 mainly pays attention to k1, slightly pays attention to k2, and does not pay attention to k3. q3 pays attention to k1, k2 and k3. This ensures that the model can only see the output at any point in time. This is in line with the causal laws of events in the real world and helps the model (molecular marker) make more reasonable and accurate predictions. For example, assuming that q2 can pay attention to k3, it may receive hints about future information, which violates the order constraints in natural language. By restricting q2 from paying attention to k3, the model will not obtain information that it should not know in advance. When the model learns to make predictions with limited information, it helps the model learn to extract and utilize key features and improve the generalization ability of the model. At the same time, this way of limiting the attention range can reduce the amount of calculation, because the model does not need to calculate the attention weights with all key vectors (corresponding to k1, k2 and k3 above) for each query vector (corresponding to q1, q2 and q3 above).

[0163] For example, in step 201 to step 204, the text features of the molecular description text and the first query vector sequence can be subjected to first attention processing through a causal attention mask to obtain first attention features. The causal attention mask ensures that when the model predicts the output at any time point, it can only see the information at that time point and before, and cannot access future information. This is in line with the causal laws of events in the real world and helps the model make more reasonable and accurate predictions.

[0164] Continue to see Figure 3B In step 10112, a second attention process is performed on the first attention feature and the encoding feature of the molecular graph data to obtain a second attention feature.

[0165] In some embodiments, the structural information of a molecule is represented as a graph (molecular graph data) using Indicates that is a set of atoms, ε represents the edge between atoms, is the number of atoms. The molecular graph data is feature encoded by a pre-trained molecular encoder to obtain the encoding features of the molecular graph data.

[0166] Taking the pre-trained molecular encoder Multi-modal molecule structure-text model for text-based retrieval and editing (MoleculeSTM) as an example, the encoding features of molecular graph data can be obtained in the following way: First, assign feature vectors to each atom and edge in the molecular graph data. Atomic features can include atom type, charge, whether it is part of a ring, etc.; edge features can include chemical bond type, bond direction, etc. Next, use the embedding layer to convert the pre-processed atomic and edge features into fixed-size embedding vectors. These vectors will be used as input in subsequent Transformer layers. Next, construct a graph attention matrix based on the embedded representation of nodes and edges. This matrix represents the degree of association between the atoms in the molecule. It passes through the Transformer encoder (including multiple self-attention layers and feedforward neural network layers). In the self-attention layer, the attention score of each node is calculated to determine the relationship between the nodes and update the node representation accordingly. In the feedforward network layer, the model further processes the representation of each node to extract more complex features. After each self-attention layer and feedforward network layer, residual connection and layer normalization are performed. After multiple Transformer encoder layers, the embedding vector of each node is updated. These vectors can be used as the final feature representation of the node, thereby obtaining the encoding features of the molecular graph data.

[0167] For example, the encoding features of molecular graph data can be expressed as Here, F represents the number of features, which means that each atom has F different features, such as atomic charge, atomic radius, electronegativity, etc. Each atomic feature contains context-aware structural information, which is used to reflect the relationship and interaction between the atom and its surrounding atoms, such as the distance, angle and torsion angle with other atoms.

[0168] In some embodiments, a second attention process is performed on the first attention feature and the encoding feature of the molecular graph data through a cross attention mechanism to obtain a second attention feature.

[0169] For example, the first attention feature is used as the query vector (Q), and the encoding feature of the molecular graph data is used as the key vector (K) and the value vector (V). The attention score is obtained by calculating the dot product of Q and K. Next, the attention score is normalized, for example, a normalization function (such as a softmax function) is applied to convert the attention score into a probability distribution as an attention weight. The encoding feature (V) of the molecular graph data is weighted using the attention weight to generate a new feature representation. The new feature representation can be linearly transformed through a linear layer to obtain a rich representation of the correlation between different positions in the encoding feature of the molecular graph data, namely, the second attention feature.

[0170] In step 10113, the second attention feature is feed-forward processed to obtain an interactive query sequence.

[0171] In some embodiments, the second attention feature is feed-forward processed through a feed-forward network layer (such as a fully connected layer or a linear layer) to obtain an interactive query sequence.

[0172] For example, the second attention feature is linearly mapped through the weight matrix and bias vector of the feedforward network layer to obtain a linear feature, and the linear feature is nonlinearly mapped through an activation function (such as sigmoid, etc.) to obtain an interactive query sequence.

[0173] For example, the interactive query sequence can be expressed as Where M represents the number of queries, which refers to the number of interactive query vectors in the interactive query sequence, that is, Z includes M interactive query vectors z i , d represents the dimension of each interactive query vector, that is, the number of elements in the interactive query vector. This dimension determines the amount of information and complexity that each interactive query vector can carry.

[0174] Continue to see Figure 3A In step 1012, the codebook of the current training batch is queried based on the interactive query sequence to obtain a molecular marker sequence.

[0175] In some embodiments, the interactive query vector sequence includes multiple interactive query vectors, see Figure 3D , Figure 3A Step 1012 shown in can be implemented by following steps 10121 to 10122, which are described in detail below.

[0176] In step 10121, the nearest encoding vector of each interactive query vector is queried from the encoding book of the current training batch, where the nearest encoding vector is the encoding vector with the smallest distance to the interactive query vector, and the index of each nearest encoding vector is queried.

[0177] In some embodiments, the codebook includes code vectors and corresponding indices, from the codebook of the current training batch In the codebook, the nearest encoding vector of each interactive query vector can be expressed by formula (1):

[0178] s i =argmin j{1，...，K} ||z i -c j ||2 (1)

[0179] Where i = 1, 2, ..., M, that is, the interactive query vector z i Quantized to a vector c in the codebook j ,s i It can be regarded as an index, that is, s i is such that ||z i -c j ||2 smallest j, for example, assuming there is an interactive query vector z1 and a codebook (c1, c2, c3), calculate ||z i -c j ||2, assuming that the distance between z1 and c2 is the smallest, that is, c2 is the closest encoding vector of z1, then s i =2.

[0180] In step 10122, the index of each most recent encoding vector is combined to obtain a molecular marker sequence.

[0181] In some embodiments, the index combination of the nearest encoding vector of each interactive query vector in the interactive query vector sequence is used as the molecular label sequence.

[0182] Continuing with the above example, the molecular representation sequence can be expressed as

[0183] Continue to see Figure 3A In step 1013, a first loss value is determined based on the interactive query sequence and the molecular marker sequence.

[0184] In some embodiments, the molecular marker sequence includes multiple molecular markers, see Figure 3E , Figure 3A Step 1013 shown can be implemented by following steps 10131 to 10134, which are described in detail below.

[0185] In step 10131, a coding vector indexed by each molecular marker is queried from the coding book of the current training batch, and the coding vector corresponding to each molecular marker is combined into a first embedded feature.

[0186] In some embodiments, the molecular marker sequence As an index, query the encoding vector in the encoding book of the current training batch, and combine the encoding vector corresponding to each molecular marker into the first embedding feature.

[0187] Continuing from the above example, the first embedding feature can be expressed as

[0188] In step 10132, feature mapping processing is performed on the first embedded feature to obtain a mapping feature.

[0189] In some embodiments, a multilayer perceptron (MLP) may be used as an adapter to perform feature mapping processing on the first embedded feature to obtain a mapping feature.

[0190] For example, through MLP (corresponding to Fig. 7A The adapter in the embodiment of the present invention performs feature mapping processing on the first embedding feature to obtain mapping features, which can be achieved in the following ways: linearly map the second attention feature through the weight matrix and bias vector of the fully connected layer in the multi-layer perceptron to obtain linear features, and perform nonlinear mapping processing on the linear features through an activation function (such as sigmoid, etc.) to obtain mapping features.

[0191] For example, you can use Fig. 7A The pre-trained molecular representation data decoder shown in performs molecular reconstruction based on the mapping features to obtain a reconstructed molecular graph.

[0192] Continuing with the above example, the mapping feature can be expressed as X R =ψ(C), where ψ represents a multilayer perceptron and C represents the first embedded feature.

[0193] In step 10133, the molecular representation data is subjected to embedded coding processing to obtain a second embedded feature.

[0194] In some embodiments, the molecular representation data is embedded and encoded using a pre-trained molecular representation data encoder (SMILES Encoder) to obtain a second embedded feature.

[0195] For example, a pre-trained molecular representation data encoder (corresponding to Fig. 7AThe molecular representation data shown in the figure is taken as an example. The molecular representation data (SMILES string) is embedded and encoded to obtain the second embedding feature, which can be achieved by the following method: SMILES string preprocessing: First, each character in the SMILES string (including atoms, bonds in chemistry, branch points, etc.) is converted into a corresponding category label or atom type, and special symbols in the SMILES string (such as brackets at branch points) are converted into special tokens that can represent molecular structures. A vocabulary is constructed containing all possible SMILES characters and tokens. This vocabulary is used to convert SMILES strings into integer indices. The SMILES string is converted into a sequence of integers using the vocabulary, each integer corresponding to a specific character or token in the vocabulary. In order to retain the position information in the sequence, position encoding is applied to the sequence. The position encoding is generated by sine and cosine functions, or it can be a learned fixed vector. The integer sequence and the positional encoding are passed as input to the encoder of ChemFormer, and the integer sequence and the positional encoding are encoded by the encoder of ChemFormer to obtain the second embedding feature.

[0196] For example, the encoder of ChemFormer consists of multiple identical layers, each of which includes the following operations: Through the self-attention mechanism, the model is able to assign different attention weights to each element in the sequence in order to capture its meaning based on the context. After self-attention, each element is passed through a feedforward network for further feature transformation. After each sublayer (self-attention and feedforward network), residual connections and layer normalization are performed to improve the stability and performance of the model. After passing through all encoder layers, each input token (corresponding to an element in the SMILES string) will have a feature vector representation, which contains the contextual information of the token, so that the vector representation of each token is combined to obtain the second embedding feature.

[0197] In step 10134, a first loss value is determined based on the mapped feature, the second embedded feature, the interactive query vector sequence, and the first embedded feature.

[0198] In some embodiments, see Figure 3F , Figure 3E The step 10134 shown can be implemented by following the steps 301 to 304, which are described in detail below.

[0199] In step 301 , an alignment loss is determined based on a difference between the mapped feature and the second embedded feature.

[0200] In some embodiments, the alignment loss is determined based on the difference between the mapped feature and the second embedded feature, which can be expressed by formula (2):

[0201]

[0202] Among them, X S represents the second embedding feature, X R Represents the mapping feature.

[0203] In step 302, a codebook loss is determined based on a first difference between the interactive query vector sequence and the first embedded feature.

[0204] In some embodiments, based on the first difference between the interactive query vector sequence and the first embedded feature, the codebook loss is determined, which can be expressed by formula (3):

[0205]

[0206] Where M represents the number of interactive query vectors in the interactive query vector sequence (i.e., the number of queries), sg[·] represents the stop gradient operator, Indicates that the molecule in the first embedded feature is labeled s i The encoding vector, z i represents the i-th interactive query vector in the interactive query vector sequence.

[0207] In step 303, a consistency loss is determined based on a second difference between the interaction query vector sequence and the first embedded feature.

[0208] In some embodiments, based on the second difference between the interactive query vector sequence and the first embedded feature, the consistency loss is determined, which can be expressed by formula (4):

[0209]

[0210] Wherein, β represents a hyperparameter, and in the embodiment of the present application, β is set to 0.25.

[0211] In step 304, a first loss value is determined based on the alignment loss, the codebook loss, and the consistency loss.

[0212] Continuing with the above example, based on the alignment loss, codebook loss and consistency loss, the first loss value is determined, which can be expressed by formula (5):

[0213]

[0214] Among them, the first term represents the alignment loss (corresponding to formula (2)), the second term is the codebook loss aimed at updating the codebook embedding (corresponding to formula (3)), and the third term is the commitment loss that encourages the interactive query vector sequence to be consistent with the embedding representation (first embedding feature) of the selected codebook (corresponding to formula (4)).

[0215] Continue to see Figure 3A In step 1014, the molecular marker, the first query vector sequence and the codebook are updated based on the first loss value, wherein the updated molecular marker, the first query vector sequence and the codebook are used for the next training batch of the molecular marker.

[0216] In some embodiments, gradient information is obtained through the first loss value, and the molecular marker, the first query vector sequence and the codebook are updated according to the gradient information to obtain updated molecular markers, the first query vector sequence and the codebook.

[0217] For example, the gradient information of the first loss value for each parameter of the molecular marker, the first query vector sequence and the code book is obtained through the back-propagation algorithm, and the parameters of the molecular marker, the first query vector sequence and the code book are updated using the obtained gradient information according to the gradient descent optimization algorithm (such as batch gradient descent, stochastic gradient descent, etc.), and the above process is repeated until a certain number of iterations is reached or the molecular marker, the first query vector sequence and the code book converge, thereby obtaining the trained molecular marker, the first query vector sequence and the code book.

[0218] For example, since the vector quantization process from the interactive query sequence to the molecular marker sequence is not differentiable, a straight-through estimation method is used to copy the gradient information from the molecular marker sequence to the interactive query sequence for gradient update.

[0219] Through steps 1011 to 1014, molecular markers are used to unify different modal data such as molecules and text into discrete token representations. This unified representation eliminates the encoding differences between different modalities, allowing the model to process and fuse multimodal information more naturally. It can more accurately capture and align the deep relationship between molecules and text, significantly improving the performance of molecular data processing in multimodal tasks.

[0220] In some embodiments, see Figure 3G The feature interaction processing is implemented by the pre-trained query converter in the molecular marker. Before iteratively executing multiple training batches on the molecular marker, multiple training batches can also be executed on the query converter, wherein the following steps 401 to 406 are executed in each training batch of the query converter, which are described in detail below.

[0221] For example, see Figure 7B , Figure 7B is a second schematic diagram of an optional structure of a molecular data processing model provided in an embodiment of the present application, Figure 7B A schematic diagram of an optional structure of a query converter is shown. Each training batch of the query converter performs three training tasks. The query converter and the second query vector sequence obtained after multiple training batches will be used for processing in step 1011.

[0222] In step 401, a text feature sample of a molecule description text sample is obtained by querying a converter.

[0223] In some embodiments, the text feature samples of the molecular description text samples can be obtained by: performing word segmentation processing on the molecular description text samples to obtain input units; performing token embedding encoding (TokenEmbeddings) processing on the input units to obtain token embedding features; performing self-attention encoding processing on the token embedding features to obtain text feature samples of the molecular description text samples.

[0224] For example, punctuation marks such as spaces, periods, and commas can be used as segmentation marks to segment the molecular description text samples, that is, to divide the molecular description text samples into input units. Third-party segmentation tools can also be used to segment the molecular description text samples, such as Jieba, NLTK, SpaCy, etc. These tools can achieve accurate Chinese and English segmentation through algorithms and language models.

[0225] For example, a word embedding model (such as Word2Vec, GloVe, etc.) can be used to perform word embedding encoding processing on the input unit, each input unit is represented as a vector of a fixed length, and word embedding features (sequences) are formed in order.

[0226] For example, the word embedding features can be used as query vectors (Query, Q), key vectors (Key, K) and value vectors (Value, V). The attention score is obtained by calculating the dot product of Q and K. Next, the attention score is normalized, for example, a normalization function (such as a softmax function) is applied to convert the attention score into a probability distribution as an attention weight. The attention weight is used to weight the word embedding features (corresponding to V above) to generate a new feature representation (corresponding to Figure 7B In the self-attention calculation of the molecular description text sample, the new feature representation can be linearly transformed through the linear layer (LinearLayer) to obtain a rich representation that includes the correlation of different positions in the word embedding feature, that is, the text feature sample of the molecular description text sample.

[0227] In step 402, a coding feature sample of a molecular graph data sample is obtained.

[0228] In some embodiments, a pre-trained molecule encoder (Molecule Encoder) is used to perform feature encoding processing on the molecule graph data sample to obtain an encoded feature sample of the molecule graph data sample.

[0229] For example, with a pre-trained molecular encoder (corresponding to Figure 7B Taking the multi-modal molecule structure-text model for text-based retrieval and editing (MoleculeSTM) as an example, the encoding features of the molecular graph data sample can be obtained in the following way: First, assign a feature vector to each atom and edge in the molecular graph data sample. The atomic features can include atom type, charge, whether it is part of a ring, etc.; edge features can include chemical bond type, bond direction, etc. Next, use the embedding layer to convert the preprocessed atomic and edge features into fixed-size embedding vectors. These vectors will be used as input in subsequent Transformer layers. Next, construct a graph attention matrix based on the embedded representation of nodes and edges. This matrix represents the degree of association between the atoms in the molecule. It passes through the Transformer encoder (including multiple self-attention layers and feedforward neural network layers). In the self-attention layer, the attention score of each node is calculated to determine the relationship between the nodes and update the node representation accordingly. In the feedforward network layer, the model further processes the representation of each node to extract more complex features. After each self-attention layer and feedforward network layer, residual connection and layer normalization are performed. After multiple Transformer encoder layers, the embedding vector of each node is updated. These vectors can be used as the final feature representation of the node, thereby obtaining the encoded feature sample of the molecular graph data sample.

[0230] In step 403, the query converter performs first attention processing on the text feature sample and the second query vector sequence of the current training batch of the query converter to obtain a first attention feature sample, wherein when the current training batch of the query converter is the first training batch, the second query vector of the current training batch of the query converter is randomly initialized.

[0231] In some embodiments, based on the text feature samples and the second query vector sequence, a first attention score matrix sample is determined (see the description of step 201 above); a mask matrix sample of the first attention score matrix sample is obtained (see the description of step 202 above); the first attention score matrix sample is masked based on the mask matrix sample, and the obtained mask score matrix sample is normalized to obtain a first attention weight matrix sample (see the description of step 203 above); the text feature sample is weighted and summed using the first attention weight matrix sample to obtain a first attention feature sample (see the description of step 204 above).

[0232] In step 404, the second attention processing is performed on the first attention feature sample and the encoding feature sample by querying the converter to obtain a second attention feature sample.

[0233] In some embodiments, a second attention process is performed on the first attention feature sample and the encoding feature sample through a cross attention mechanism (corresponding to Figure 7B Cross attention shown in ), a second attention feature sample is obtained. The specific implementation method can be found in the description of step 10112 above, which will not be repeated here.

[0234] In step 405, the second attention feature sample is feed-forward processed by the query converter to obtain an interactive query sequence sample.

[0235] In some embodiments, the second attention feature sample is feedforward processed through a feedforward network layer (such as a fully connected layer or a linear layer) to obtain an interactive query sequence sample. The specific implementation method can be found in the description of step 10113 above and will not be repeated here.

[0236] In step 406, multiple training tasks are performed based on the interactive query sequence samples and the text feature samples to obtain a pre-trained query converter.

[0237] In some embodiments, the plurality of training tasks include a molecular text comparison learning training task, a molecular text matching training task, and a molecular text generation training task, see Figure 3H , Figure 3G The step 406 shown can be implemented by following the steps 4061 to 4065, which are described in detail below.

[0238] In step 4061, based on the interactive query sample and the text feature sample, a first sub-loss value corresponding to the molecular text contrastive learning training task is determined.

[0239] In some embodiments, based on the interactive query sample and the text feature sample, the first sub-loss value corresponding to the Molecule-Text Contrastive Learning (MTC) training task is determined, which can be expressed by formula (6):

[0240]

[0241] Among them, B represents the batch size, and τ represents the temperature parameter (hyperparameter). Here, and yi refer to the i-th interactive query sample and text feature sample in the same training batch, respectively.

[0242] Here, the molecule-text contrastive learning training task (MTC) aims to align the molecular and text features by maximizing the information between them. This is achieved by maximizing the molecule-text similarity of the positive sample pairs and the similarity of the negative sample pairs. The last interactive query vector sample z in the interactive query sample is used M As a representation of the query, the output interaction query samples (a sequence consisting of multiple interaction query vector samples) are causal, that is, the last interaction query vector sample contains the global information from the interaction query samples.

[0243] In step 4062, a text embedding feature sample of the molecule description text sample is obtained.

[0244] In some embodiments, the text embedding feature samples of the molecular description text samples can be obtained by: performing word segmentation processing on the molecular description text samples to obtain input units; performing token embedding coding (TokenEmbeddings) processing on the input units to obtain text embedding feature samples of each word.

[0245] In step 4063, based on the interactive query sample, the encoding feature sample and the text embedding feature sample, a second sub-loss value corresponding to the molecular text matching training task is determined.

[0246] In some embodiments, based on the interactive query sample, the encoding feature sample and the text embedding feature sample, the second sub-loss value corresponding to the Molecule-Text Matching (MTM) training task is determined, which can be expressed by formula (7):

[0247]

[0248] Among them, φ represents a binary classifier, X i and t iThey refer to the encoded feature sample of the i-th input in a training batch and the input text embedding feature sample respectively, and the last interactive query vector sample z in the interactive query sample M As a representation of a query.

[0249] Specifically, due to the interactive query sequence sample The information of molecular modality and text modality are captured by the first attention process and the second attention process respectively, and the last interactive query vector sample z in the interactive query sample is used M As input to a binary classifier, this classifier predicts whether a given molecule-text pair matches or not.

[0250] In step 4064, based on the text embedding feature samples and the encoding feature samples, the third sub-loss value corresponding to the molecular text generation training task is determined.

[0251] In some embodiments, based on the text embedding feature samples and the encoding feature samples, a third sub-loss value corresponding to the Molecule-grounded Text Generation (MTG) training task is determined. Specifically, MTG refers to generating a molecular text description sample given an encoding feature sample input. In this task, since only text output is required, the causal mask is not applied to the second query vector sequence, but the causal mask is applied to the text embedding feature samples, that is, each text token (the text embedding feature sample of each word) is allowed to pay attention to the previous text token and all second query vector sequences, but not to the subsequent text tokens (corresponding to Figure 7B Different training tasks in the language modeling (LM) loss function is used to model the i Under the condition of generating t i , expressed as formula (8):

[0252]

[0253] in, Represents text embedding feature sample t i The jth text token in .

[0254] In step 4065, based on the first sub-loss value, the second sub-loss value and the third sub-loss value, the query converter and the second query vector sequence of the current training batch are updated, wherein the updated query converter and the second query vector sequence are used for the next batch of training of the query converter, and the second query vector sequence updated in the last training batch is used as the first query vector sequence of the first training batch of the molecular marker.

[0255] In some embodiments, a total loss value for training the query converter is determined based on the first sub-loss value, the second sub-loss value, and the third sub-loss value, which can be expressed by formula (9):

[0256] L Q-Former =L MTC +L MTM +L MTG (9)

[0257] In some embodiments, gradient information is obtained through the total loss value, and the query converter and the second query vector sequence of the current training batch are updated according to the gradient information to obtain an updated query converter and the second query vector sequence.

[0258] For example, the gradient information of the total loss value for each parameter of the query converter and the second query vector sequence is obtained through the back-propagation algorithm, and the parameters of the query converter and the second query vector sequence are updated using the obtained gradient information according to the gradient descent optimization algorithm (such as batch gradient descent, stochastic gradient descent, etc.). The above process is repeated until a certain number of iterations is reached or the query converter and the second query vector sequence converge, thereby obtaining the trained query converter and the second query vector sequence.

[0259] Continue to see Figure 3A In step 102, based on the molecular markers after multiple training batches, the training task of the molecular data processing model is performed.

[0260] In some embodiments, see Fig. 3I , Figure 3A The step 102 shown can be implemented by following the steps 501 to 502, which are described in detail below.

[0261] In step 501, based on the molecular tag sequence, the molecular representation data and the prompt, a first autoregressive training task is performed on the pre-trained language understanding model to obtain a first language understanding model for generating molecular description text.

[0262] In some embodiments, see Figure 3J , Figure 3A The step 501 shown can be implemented by following the steps 5011 to 5015, which are described in detail below.

[0263] In step 5011, the molecular tag sequence, the embedded representation of the molecular representation data, the embedded representation of the prompt and the embedded representation of the molecular description text are connected to obtain a first multimodal sequence.

[0264] In some embodiments, the molecular marker sequence, the embedded representation of the molecular representation data, the embedded representation of the prompt, and the embedded representation of the molecular description text are concatenated together to obtain a first multimodal sequence.

[0265] For example, the molecular marker sequence can be Figure 7C The molecular marker shown in FIG. 5 (the parameters remain unchanged in the training of step 501) processes the molecular graph data to obtain the embedded representation of the molecular representation data and the embedded representation of the prompt words. Figure 7C The text tokenizer (Tokenizer) mapping shown in is obtained.

[0266] For example, to facilitate the representation of multimodal sequences, a molecular vocabulary is constructed The molecular vocabulary maintains the order of the codebook For example, if c1 in the codebook is the representation of a molecule, then is the corresponding discrete mark. Some special markers are also included, such as boundary indicators, such as [MOL] and [ / MOL], to mark the beginning and end of the molecule sequence. and Molecular Glossary Synthesis into a unified molecule - text vocabulary It helps to jointly learn molecules and text under a unified next token prediction objective. As the vocabulary expands, the corresponding embedding and prediction layers also need to be expanded, and the newly introduced parameters are initialized using a random initialization method.

[0267] In step 5012, the embedded representation of the molecular description text in the first multimodal sequence is masked to obtain a first input sequence.

[0268] In some embodiments, the embedding representation of the molecular description text in the first multimodal sequence is masked, that is, the embedding representation of the molecular description text is used as the generation target of the first autoregressive training task to obtain the first input sequence.

[0269] In step 5013, a first prediction process is performed on the embedded representation of the masked molecule description text in the first input sequence through the pre-trained language understanding model to obtain a first prediction result.

[0270] In some embodiments, a pre-trained language understanding model (e.g., a large language model) is used to predict the embedded representation of the masked molecular description text, and the masked part (corresponding to Figure 7C), this process may include using the encoder and decoder of the language understanding model to generate a first prediction result. For example, the language understanding model uses its encoder to process the unmasked portion of the first input sequence and calculates the hidden state of each position. For the embedded representation of the masked molecular description text, the language understanding model predicts the original molecular description text based on the encoded context information (the hidden state of each position). The language understanding model applies a softmax function through the decoder to generate a probability distribution (indicating the probability value of each word of the predicted molecular description text) as the first prediction result, or outputs the predicted molecular description text as the first prediction result (corresponding to Figure 7C The molecular description text output by the first language understanding model shown in ).

[0271] In step 5014, a first prediction loss value is determined based on the first prediction result and the embedded representation of the molecule description text.

[0272] In some embodiments, the first prediction loss value is determined according to the first prediction result and the embedded representation of the molecule description text, which can be expressed by formula (10):

[0273]

[0274] in, Representative dataset (multimodal data used), represents the index set of the generated target (embedded representation of the molecule description text), and θ represents the parameters of the language understanding model. For each element u in the sequence i , the model will calculate given all the previous elements u1,…,u i-1 Time i The conditional probability p of , then logarithm and sum. The unified representation and training paradigm of molecules and text enhances the ability of language understanding models to understand the interaction and alignment between molecules and text.

[0275] In step 5015, the pre-trained language understanding model is updated based on the first prediction loss value to obtain a first language understanding model for generating molecular description text.

[0276] In some embodiments, gradient information is obtained through the first prediction loss value, and the pre-trained language understanding model is updated according to the gradient information to obtain an updated first language understanding model.

[0277] For example, the gradient information of the first prediction loss value for each parameter of the language understanding model is obtained through the back-propagation algorithm, and the parameters of the language understanding model are updated using the obtained gradient information according to the gradient descent optimization algorithm (such as batch gradient descent, stochastic gradient descent, etc.). The above process is repeated until a certain number of iterations is reached or the language understanding model converges, thereby obtaining the trained first language understanding model.

[0278] Continue to see Fig. 3I In step 502, based on the molecular marker sequence, the prompt and the molecular description text, the first language understanding model is subjected to a second autoregressive training task to obtain a second language understanding model for generating molecular description text and for generating molecular marker sequences, wherein the second language understanding model serves as a molecular data processing model.

[0279] In some embodiments, see Figure 3K , Fig. 3I The step 502 shown can be implemented by following the steps 5021 to 5025, which are described in detail below.

[0280] In step 5021, the molecular marker sequence, the embedded representation of the prompt, and the embedded representation of the molecular description text are connected to obtain a second multimodal sequence.

[0281] Here, the specific implementation method can refer to the description of step 5011 above, which will not be repeated here.

[0282] In step 5022, the molecular marker sequence in the second multimodal sequence is masked to obtain a second input sequence.

[0283] In some embodiments, the molecular marker sequence in the first second multimodal sequence is masked, that is, the molecular marker sequence is used as the generation target of the second autoregressive training task (corresponding to Fig.7D The mask shown in ) is used to obtain the second input sequence.

[0284] In step 5023, a second prediction process is performed on the molecular marker sequence through the first language understanding model to obtain a second prediction result.

[0285] In some embodiments, the masked molecular marker sequence is predicted by a pre-trained language understanding model (e.g., a large language model) to restore the masked portion. This process may include using the encoder and decoder of the language understanding model to generate a second prediction result. For specific implementation methods, please refer to the description of step 5013. The prediction processing of the two is similar, and the difference lies in the generation of different targets.

[0286] For example, the predicted molecular marker sequence (corresponding to Fig.7DThe molecular tag sequence output by the second language understanding model shown in ) can be used for molecular reconstruction to obtain a reconstructed molecule (corresponding to Fig.7D Model reasoning shown in ).

[0287] In step 5024, a second prediction loss value is determined based on the second prediction result and the molecular marker sequence.

[0288] In some embodiments, the second prediction loss value is determined based on the second prediction result and the molecular marker sequence as described in formula (10) above, that is, the generation target in formula (10) is converted into a molecular marker sequence.

[0289] In step 5025, the first language understanding model is updated based on the second prediction loss value to obtain a second language understanding model for generating molecular description text and for generating molecular marker sequences.

[0290] Here, the specific implementation method can refer to the description of step 5015 above, which will not be repeated here.

[0291] Through step 501 to step 502, using the autoregressive training model, not only can the molecular description text be generated, but also the molecular label sequence can be generated. This unified generation framework supports the joint generation and understanding of multimodal information, providing a more flexible and universal generation method. In the fields of drug discovery and material design, it is possible to generate molecules with specific properties and generate related text descriptions at the same time, thereby achieving the beneficial effect of accelerating the discovery and development of new molecules.

[0292] In some embodiments, see Figure 3L , Figure 3A The step 102 shown can be implemented by following the steps 503 to 504, which are described in detail below.

[0293] In step 503, based on the molecular tag sequence, the molecular representation data and the prompt, a first autoregressive training task is performed on the pre-trained language understanding model to obtain a first molecular data processing model for generating molecular description text.

[0294] Here, the specific implementation method can refer to the description of steps 5011 to 5015 above. The first language understanding model corresponds to the first molecular data processing model, which will not be repeated here.

[0295] In step 504, based on the molecular marker sequence, the prompt and the molecular description text, a second autoregressive training task is performed on the pre-trained language understanding model to obtain a second molecular data processing model for generating a molecular marker sequence.

[0296] Here, the specific implementation method can refer to the description of steps 5021 to 5025 above. A second autoregressive training task is performed on the basis of the pre-trained language understanding model, and the parameters of the pre-trained language understanding model are updated to obtain a second molecular data processing model for generating a molecular marker sequence.

[0297] In some embodiments, see Figure 3M After executing the training task of the molecular data processing model based on the trained molecular marker, the following steps 601 to 604 may also be executed, which are described in detail below.

[0298] In step 601, training data for a specific task related to molecular processing is obtained, wherein the training data for the specific task includes molecular data samples and labeled data samples.

[0299] In some embodiments, specific tasks related to molecule processing may include: molecular property prediction, which is used to predict various physical, chemical or biological activity properties of molecules, such as solubility, activity, stability, etc. of molecules); molecular naming, which is used to convert the structural information of molecules into chemical names or output text descriptions of molecules; molecular text retrieval, which is used to extract and retrieve information related to specific molecules from text data, for example, it may include searching for descriptions, properties or related research of molecules from scientific literature, patents or online databases; molecule generation, which is used to generate corresponding molecular structures based on the text description of molecules; reagent prediction, which is used to predict the chemical reagents that may be used in a specific reaction, which can help chemists choose appropriate reagents to optimize reaction pathways or improve yields; forward reaction prediction, which is used to predict the chemical reactions that may occur under specific conditions for given reactants, which can help chemists design new synthesis routes or optimize existing reactions; retrosynthesis, which is a method for reversely inferring possible synthesis pathways starting from target molecules, can be used in medicinal chemistry to help chemists find effective methods for synthesizing target molecules.

[0300] Taking the specific task of molecular property prediction as an example, the training data for the specific task may include molecular data samples (such as molecular graph data samples, molecular description text samples, and molecular representation data samples, etc.) and labeled data samples (such as molecular solubility, molecular stability, etc.).

[0301] In step 602, a molecular data processing model is used to perform specific task result prediction processing on the molecular data sample to obtain predicted data.

[0302] In some embodiments, the molecular data samples are processed by a molecular data processing model (second language understanding model), and based on the learned ability to understand the interaction and alignment between molecules and texts, the molecular data samples are processed for specific task result prediction to obtain predicted data.

[0303] Taking the specific task of molecular property prediction as an example, the predicted data may include solubility (unit: mol / L or g / L), decomposition temperature, half-life, reaction rate constant, etc.

[0304] In step 603, a specific task loss value is determined based on the labeled data samples and the predicted data.

[0305] In some embodiments, the difference value between the labeled data sample and the predicted data is obtained through a preset loss function (such as a cross entropy loss function) as a specific task loss value.

[0306] In step 604, the molecular data processing model is updated based on the specific task loss value to obtain a molecular data processing model for the specific task.

[0307] In some embodiments, gradient information is obtained through a specific task loss value, and the molecular data processing model is updated according to the gradient information to obtain an updated molecular data processing model for the specific task.

[0308] For example, the gradient information of the loss value of a specific task for each parameter of the molecular data processing model is obtained through the back-propagation algorithm, and the parameters of the molecular data processing model are updated using the obtained gradient information according to the gradient descent optimization algorithm (such as batch gradient descent, stochastic gradient descent, etc.). The above process is repeated until a certain number of iterations is reached or the molecular data processing model converges, thereby obtaining a trained molecular data processing model for a specific task.

[0309] For example, low-rank adaptation (LoRA) can be used to introduce additional low-rank matrix parameters in certain network layers of the molecular data processing model. The number of these low-rank matrix parameters is relatively small, so it will not significantly increase the computational and storage burden. Therefore, for specific tasks, only the parameters of these low-rank matrices are trained instead of the parameters of the entire pre-trained model, thereby improving training efficiency.

[0310] Through steps 601 to 604, the multimodal unified representation and generation capabilities of the trained molecular data processing model are utilized to make it widely applicable in a variety of application scenarios (specific tasks). Whether it is molecule-text generation, molecular property prediction, or cross-modal information retrieval, effective solutions can be provided. Specifically, the unified architecture design of the molecular data processing model simplifies the integration and deployment of the system, enhances the adaptability of the model in different tasks and fields, and solves the limitations of traditional models in multimodal data processing through unified representation and generation methods, and improves the performance of the molecular data processing model in various multimodal tasks. It not only enhances the performance of the molecular data processing model, but also expands the boundaries of multimodal data analysis and generation, and provides new tools and methods for research and application in related fields.

[0311] See also Figure 4 , Figure 4 It is a first flow chart of the molecular data processing method provided in an embodiment of the present application.

[0312] In step 701, molecular data to be processed and prompt words to be processed are obtained.

[0313] In some embodiments, the molecular data to be processed can be obtained from molecular databases, such as PubChem, DrugBank, etc. These databases provide a large amount of molecular structure and related chemical information. In interactive applications, prompts to be processed are obtained through specific queries or instructions input by the user, and predefined prompts can also be obtained as prompts to be processed. These predetermined prompts can be used to guide the molecular data processing model to perform specific tasks, such as generating molecular descriptions, predicting molecular properties, etc. When processing a dialogue system or a question-and-answer system, the prompts to be processed can also be extracted from the history of the dialogue to provide contextual information. The embodiments of the present application do not limit the specific implementation methods of obtaining the molecular data to be processed and the prompts to be processed.

[0314] In step 702, in response to the molecular data to be processed representing a molecular structure, a molecular marker is used to generate a molecular marker for the molecular data to be processed, thereby obtaining a molecular marker sequence for the molecular data to be processed.

[0315] In some embodiments, when the molecular data to be processed represents a molecular structure (represented as a molecular graph), a molecular marker is used to generate molecular markers for the molecular data to be processed to obtain a molecular marker sequence of the molecular data to be processed.

[0316] For example, a first attention processing (such as causal attention) is performed on the learnable query vector sequence obtained after training (corresponding to the first query vector sequence after the last training batch is updated) to obtain a first attention feature (the specific implementation method can be seen in the description of step 10111 above); a second attention processing is performed on the first attention feature and the encoding feature of the molecular data to be processed to obtain a second attention feature (see the description of step 10112 above); the second attention feature is feed-forward processed to obtain an interactive query sequence (see the description of step 10113 above); from the encoding book obtained after training, the nearest encoding vector of each interactive query vector is queried, wherein the nearest encoding vector is the encoding vector with the smallest distance to the interactive query vector, and the index of each nearest encoding vector is queried (see the description of step 10121 above); the index of each nearest encoding vector is combined to obtain a molecular labeling sequence of the molecular data to be processed (see the description of step 10122 above).

[0317] For example, the learnable query vector sequence obtained after training and the coding features of the molecular data to be processed are subjected to second attention processing to obtain the second attention features (see the description of step 10112 above); the second attention features are feed-forward processed to obtain an interactive query sequence (see the description of step 10113 above); from the coding book obtained after training, the nearest coding vector of each interactive query vector is queried, wherein the nearest coding vector is the coding vector with the smallest distance to the interactive query vector, and the index of each nearest coding vector is queried (see the description of step 10121 above); the index of each nearest coding vector is combined to obtain a molecular marker sequence of the molecular data to be processed (see the description of step 10122 above).

[0318] In step 703, molecular description text prediction processing is performed on the molecular data processing model based on the molecular marker sequence of the molecular data to be processed and the embedded representation of the prompt to be processed to obtain the molecular description text of the molecular data to be processed, wherein the molecular marker and the molecular data processing model are trained by the training method of the molecular data processing model provided in the embodiment of the present application.

[0319] In some embodiments, a molecular data processing model is used (for the training of the molecular data processing model, see the description of steps 501 to 502 above) to perform molecular description text prediction processing based on the molecular marker sequence of the molecular data to be processed and the embedded representation of the prompt to be processed, thereby obtaining the molecular description text of the molecular data to be processed.

[0320] For example, the molecular description text of the molecular data to be processed is predicted by the molecular data processing model. This process may include using the encoder and decoder of the molecular data processing model to generate prediction results. For example, the molecular data processing model uses its encoder to process the molecular tag sequence of the molecular data to be processed and the embedded representation of the prompt to be processed, and calculates the hidden state of each position. The molecular data processing model predicts the molecular description text of the molecular data to be processed based on the encoded context information (the hidden state of each position).

[0321] See also Figure 5 , Figure 5 It is a second flow chart of the molecular data processing method provided in an embodiment of the present application.

[0322] In step 801, molecular data to be processed and prompt words to be processed are obtained.

[0323] In some embodiments, the molecular data to be processed can be extracted from text resources such as scientific literature, patents, and reports through text mining and natural language processing techniques. In interactive applications, the prompts to be processed are obtained through specific queries or instructions input by the user, and predefined prompts can also be obtained as prompts to be processed. These predefined prompts can be used to guide the molecular data processing model to perform specific tasks, such as generating molecular descriptions, predicting molecular properties, etc. When processing a dialogue system or a question-and-answer system, the prompts to be processed can also be extracted from the history of the dialogue to provide context information. The embodiments of this application do not limit the specific implementation methods of obtaining the molecular data to be processed and the prompts to be processed.

[0324] In step 802, in response to the fact that the molecular data to be processed is text data, molecular marker prediction processing is performed through a molecular data processing model based on an embedded representation of the molecular data to be processed and an embedded representation of a prompt to be processed, so as to obtain a molecular marker sequence of the molecular data to be processed, wherein the molecular marker and the molecular data processing model are trained by the training method of the molecular data processing model provided in an embodiment of the present application.

[0325] In some embodiments, a molecular data processing model is used (for the training of the molecular data processing model, see the description of steps 501 to 502 above) to perform molecular marker prediction processing based on the embedded representation of the molecular data to be processed and the embedded representation of the prompt to be processed, to obtain a molecular marker sequence of the molecular data to be processed.

[0326] For example, a molecular data processing model is used to predict a molecular marker sequence of the molecular data to be processed. This process may include using an encoder and a decoder of the molecular data processing model to generate a prediction result. For example, the molecular data processing model uses its encoder to process an embedded representation of the molecular data to be processed and an embedded representation of the prompt to be processed, and calculates a hidden state at each position. The molecular data processing model predicts a molecular marker sequence of the molecular data to be processed based on the encoded context information (the hidden state at each position).

[0327] It should be noted that the embodiments of the present application can be applied to various molecular data processing scenarios, such as molecular property prediction, molecular naming, molecular text retrieval, molecular generation, reagent prediction, and forward reaction prediction. The following will describe an exemplary application of the embodiments of the present application in the molecular property prediction scenario. Figure 6 , Figure 6 is a flow chart of a molecular data processing method applied in a molecular property prediction scenario provided in an embodiment of the present application, which will be combined with Figure 6 The steps shown are explained.

[0328] In step 901, molecular data of properties to be predicted are obtained.

[0329] In some embodiments, the molecular data of the properties to be predicted can be obtained from a molecular database such as PubChem, DrugBank, etc. through prompts input by the user. These databases provide a large amount of molecular structure and related chemical information.

[0330] For example, the prompt input by the user may be "I want to know the solubility of this molecule". In the interactive system, the system further obtains the molecular data of the property to be predicted based on the above. For example, the system displays "Which molecule did you mention? Please provide its compound identifier (Compound Identifier, CID) or SMILES string" on the interactive interface, and the user provides the prompt "CID 987654321".

[0331] In step 902, molecular data to be predicted are processed for molecular property prediction using a molecular data processing model for a specific task to obtain predicted properties, wherein the molecular data processing model for the specific task is trained using the training method for the molecular data processing model provided in an embodiment of the present application.

[0332] In some embodiments, a molecular data processing model for a specific task (the specific training process can be found in the description of steps 601 to 604 above) is used to perform molecular property prediction processing based on the given molecular data of the property to be predicted, and output a number [0, 1] as a prediction of the strength of the property based on the property to be predicted. For example, "0" indicates poor stability, and "1" indicates strong stability.

[0333] The following is a description of an exemplary structure of a training device 133 for a molecular data processing model provided in an embodiment of the present application implemented as a software module. In some embodiments, Figure 2A As shown, the software modules stored in the training device 133 of the molecular data processing model in the memory 130-1 may include:

[0334] The first training module 1331 is used to iteratively execute multiple training batches for the molecular marker, wherein the following processing is performed in each training batch: feature interaction processing is performed on the molecular description text, the molecular graph data and the first query vector sequence of the current training batch by the molecular marker to obtain an interactive query sequence, the encoding book of the current training batch is queried based on the interactive query sequence to obtain a molecular marker sequence, a first loss value is determined based on the interactive query sequence and the molecular marker sequence, and the molecular marker, the first query vector sequence and the encoding book are updated based on the first loss value, wherein the updated molecular marker, the first query vector sequence and the encoding book are used for the next training batch of the molecular marker.

[0335] The second training module 1332 is used to perform the training task of the molecular data processing model based on the molecular markers after the multiple training batches.

[0336] In some embodiments, the first training model 1331 is also used to perform a first attention processing on the text features of the molecular description text and the first query vector sequence to obtain a first attention feature; perform a second attention processing on the first attention feature and the encoding features of the molecular graph data to obtain a second attention feature; and perform feedforward processing on the second attention feature to obtain the interactive query sequence.

[0337] In some embodiments, the first training model 1331 is also used to determine a first attention score matrix based on the text features of the molecular description text and the first query vector sequence; obtain a mask matrix of the first attention score matrix; mask the first attention score matrix based on the mask matrix, normalize the obtained mask score matrix to obtain a first attention weight matrix; and perform weighted summation processing on the text features through the first attention weight matrix to obtain the first attention features.

[0338] In some embodiments, the interaction query vector sequence includes multiple interaction query vectors, and the first training model 1331 is also used to query the nearest encoding vector of each of the interaction query vectors from the encoding book of the current training batch, wherein the nearest encoding vector is the encoding vector with the smallest distance to the interaction query vector, and query the index of each of the nearest encoding vectors; the index of each of the nearest encoding vectors is combined to obtain the molecular marker sequence.

[0339] In some embodiments, the molecular marker sequence includes multiple molecular markers, and the first training model 1331 is also used to query the encoding vector indexed by each molecular marker from the encoding book of the current training batch, and combine the encoding vectors corresponding to each molecular marker into a first embedded feature; perform feature mapping processing on the first embedded feature to obtain a mapping feature; perform embedded coding processing on the molecular representation data to obtain a second embedded feature; and determine a first loss value based on the mapping feature, the second embedded feature, the interactive query vector sequence and the first embedded feature.

[0340] In some embodiments, the first training model 1331 is further used to determine an alignment loss based on a difference between the mapping feature and the second embedded feature; determine a coding loss based on a first difference between the interaction query vector sequence and the first embedded feature; determine a consistency loss based on a second difference between the interaction query vector sequence and the first embedded feature; and determine a first loss value based on the alignment loss, the coding loss and the consistency loss.

[0341] In some embodiments, the feature interaction processing is implemented by a pre-trained query converter in the molecular marker, and the first training model 1331 is also used to execute multiple training batches for the query converter, wherein the following processing is performed in each training batch of the query converter: obtaining text feature samples of molecular description text samples through the query converter; obtaining encoding feature samples of molecular graph data samples; performing first attention processing on the text feature samples and the second query vector sequence of the current training batch of the query converter through the query converter to obtain first attention feature samples, wherein when the current training batch of the query converter is the first training batch, the second query vector of the current training batch of the query converter is randomly initialized; performing second attention processing on the first attention feature samples and the encoding feature samples through the query converter to obtain second attention feature samples; performing feed-forward processing on the second attention feature samples through the query converter to obtain interactive query sequence samples; performing multiple training tasks based on the interactive query sequence samples and the text feature samples to obtain the pre-trained query converter.

[0342] In some embodiments, the multiple training tasks include a molecular text contrast learning training task, a molecular text matching training task, and a molecular text generation training task. The first training model 1331 is further used to determine a first sub-loss value corresponding to the molecular text contrast learning training task based on the interactive query sample and the text feature sample; obtain a text embedding feature sample of the molecular description text sample; determine a second sub-loss value corresponding to the molecular text matching training task based on the interactive query sample, the encoding feature sample, and the text embedding feature sample; determine a third sub-loss value corresponding to the molecular text generation training task based on the text embedding feature sample and the encoding feature sample; based on the first sub-loss value, the second sub-loss value, and the third sub-loss value, update the query converter and the second query vector sequence of the current training batch, wherein the updated query converter and the second query vector sequence are used for the training of the next batch of the query converter, and the second query vector sequence updated in the last training batch is used as the first query vector sequence of the first training batch of the molecular marker.

[0343] In some embodiments, the second training module 1332 is further used to perform a first autoregressive training task on a pre-trained language understanding model based on the molecular marker sequence, the molecular representation data and the prompt to obtain a first language understanding model for generating the molecular description text; based on the molecular marker sequence, the prompt and the molecular description text, perform a second autoregressive training task on the first language understanding model to obtain a second language understanding model for generating the molecular description text and for generating the molecular marker sequence, wherein the second language understanding model serves as the molecular data processing model.

[0344] In some embodiments, the second training module 1332 is further used to connect the molecular marker sequence, the embedded representation of the molecular representation data, the embedded representation of the prompt and the embedded representation of the molecular description text to obtain a first multimodal sequence; mask the embedded representation of the molecular description text in the first multimodal sequence to obtain a first input sequence; perform a first prediction process on the masked embedded representation of the molecular description text in the first input sequence through the pre-trained language understanding model to obtain a first prediction result; determine a first prediction loss value based on the first prediction result and the embedded representation of the molecular description text; and update the pre-trained language understanding model based on the first prediction loss value to obtain the first language understanding model for generating the molecular description text.

[0345] In some embodiments, the second training module 1332 is further used to connect the molecular marker sequence, the embedded representation of the prompt and the embedded representation of the molecular description text to obtain a second multimodal sequence; mask the molecular marker sequence in the second multimodal sequence to obtain a second input sequence; perform a second prediction process on the molecular marker sequence through the first language understanding model to obtain a second prediction result; determine a second prediction loss value based on the second prediction result and the molecular marker sequence; and update the first language understanding model based on the second prediction loss value to obtain the second language understanding model used to generate the molecular description text and to generate the molecular marker sequence.

[0346] In some embodiments, the second training module 1332 is further used to perform a first autoregressive training task on a pre-trained language understanding model based on the molecular marker sequence, the molecular representation data and the prompt to obtain a first molecular data processing model for generating the molecular description text; and to perform a second autoregressive training task on the pre-trained language understanding model based on the molecular marker sequence, the prompt and the molecular description text to obtain a second molecular data processing model for generating the molecular marker sequence.

[0347] In some embodiments, the second training module 1332 is also used to obtain training data for a specific task related to molecular processing, wherein the training data for the specific task includes molecular data samples and labeled data samples; using the molecular data processing model, performing specific task result prediction processing on the molecular data samples to obtain predicted data; determining a specific task loss value based on the labeled data samples and the predicted data; and updating the molecular data processing model based on the specific task loss value to obtain a molecular data processing model for the specific task.

[0348] The following is a description of an exemplary structure of the molecular data processing device 134 provided in the present application as a software module. In some embodiments, Figure 2B As shown, the software modules stored in the molecular data processing device 134 of the memory 130-2 may include:

[0349] The data processing module 1341 is used to obtain the molecular data to be processed and the prompt words to be processed.

[0350] In some embodiments, the data processing module 1341 is further configured to, in response to the molecular data to be processed representing a molecular structure, perform molecular marker generation processing on the molecular data to be processed by a molecular marker to obtain a molecular marker sequence of the molecular data to be processed.

[0351] The prediction module 1342 is used to perform molecular description text prediction processing based on the molecular marker sequence of the molecular data to be processed and the embedded representation of the prompt to be processed through a molecular data processing model to obtain the molecular description text of the molecular data to be processed, wherein the molecular marker and the molecular data processing model are trained by the training method of the molecular data processing model provided in the embodiment of the present application.

[0352] The following is a description of an exemplary structure of the molecular data processing device 135 provided in the present application as a software module. In some embodiments, Figure 2C As shown, the software modules stored in the molecular data processing device 135 of the memory 130-3 may include:

[0353] The data processing module 1351 is used to obtain the molecular data to be processed and the prompt words to be processed.

[0354] The prediction module 1352 is used to perform molecular marker prediction processing based on the embedded representation of the molecular data to be processed and the embedded representation of the prompt to be processed through the molecular data processing model in response to the molecular data to be processed being text data, so as to obtain the molecular marker sequence of the molecular data to be processed, wherein the molecular marker and the molecular data processing model are trained by the training method of the molecular data processing model provided in the embodiment of the present application.

[0355] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device executes the training method of the molecular data processing model or the molecular data processing method described in the embodiment of the present application.

[0356] The present application embodiment provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions or computer programs are stored. When the computer-executable instructions or computer programs are executed by a processor, the processor will be caused to execute the training method of the speech synthesis model or the speech synthesis method provided in the present application embodiment, for example, Figure 3A The training method of the molecular data processing model shown or Figure 4 The molecular data processing method is shown.

[0357] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0358] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0359] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0360] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0361] To summarize, through the embodiments of the present application, multiple training batches are executed on the molecular marker, feature interaction processing is performed on the molecular graph data and the molecular description text, the first query vector sequence and the code book are continuously updated, and the interactive query sequence obtained by the feature interaction processing is used to map the molecular graph data to a molecular marker sequence, so that the representation of the molecular representation sequence and the molecular description text are aligned, thereby improving the consistency of the multimodal data representation. This unified representation eliminates the encoding differences between different modalities, so that the trained molecular data processing model can more naturally process and fuse multimodal information, accurately capture and align the deep relationship between molecules and text, and achieve the beneficial effect of improving the generalization ability of the molecular data processing model.

[0362] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent replacements and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for training a molecular data processing model, characterized in that: The molecular data processing model is trained with the assistance of molecular markers; the method comprises: A plurality of training batches are iteratively performed on the molecular marker, wherein the following processes are performed in each training batch: The molecular marker performs feature interactive processing on the molecular description text, the molecular graph data and the first query vector sequence of the current training batch to obtain an interactive query sequence, Based on the interactive query sequence, the codebook of the current training batch is queried to obtain a molecular marker sequence, Determining a first loss value based on the interactive query sequence and the molecular marker sequence, Update the molecular marker, the first query vector sequence, and the codebook based on the first loss value, wherein the updated molecular marker, the first query vector sequence, and the codebook are used for a next training batch of the molecular marker; Based on the molecular markers after the multiple training batches, the training task of the molecular data processing model is performed.

2. The method according to claim 1, characterized in that The method of performing feature interactive processing on the molecular description text, the molecular graph data and the first query vector sequence of the current training batch by the molecular marker to obtain an interactive query sequence includes: Performing first attention processing on the text features of the molecule description text and the first query vector sequence to obtain first attention features; Performing a second attention process on the first attention feature and the encoding feature of the molecular graph data to obtain a second attention feature; The second attention feature is feed-forward processed to obtain the interactive query sequence.

3. The method according to claim 2, characterized in that The performing first attention processing on the text features of the molecular description text and the first query vector sequence to obtain first attention features includes: Determining a first attention score matrix based on text features of the molecule description text and the first query vector sequence; Obtaining a mask matrix of the first attention score matrix; Performing mask processing on the first attention score matrix based on the mask matrix, and normalizing the obtained mask score matrix to obtain a first attention weight matrix; The text features are weighted and summed using the first attention weight matrix to obtain the first attention feature.

4. The method according to claim 1, characterized in that: The interactive query vector sequence includes a plurality of interactive query vectors, and querying the codebook of the current training batch based on the interactive query sequence to obtain a molecular marker sequence includes: Querying the nearest encoding vector of each of the interactive query vectors from the encoding book of the current training batch, wherein the nearest encoding vector is an encoding vector having the smallest distance with the interactive query vector, and querying the index of each of the nearest encoding vectors; The molecular marker sequence is obtained by combining the index of each of the most recent encoding vectors.

5. The method according to any one of claims 1 to 4, characterized in that: The molecular marker sequence includes a plurality of molecular markers; and determining the first loss value based on the interactive query sequence and the molecular marker sequence includes: Querying a coding vector with each of the molecular markers as an index from the coding book of the current training batch, and combining the coding vectors corresponding to each of the molecular markers into a first embedding feature; Performing feature mapping processing on the first embedded features to obtain mapping features; Performing embedding coding processing on the molecular representation data to obtain a second embedding feature; A first loss value is determined based on the mapped feature, the second embedded feature, the interaction query vector sequence, and the first embedded feature.

6. The method according to claim 5, characterized in that The determining a first loss value based on the mapping feature, the second embedding feature, the interactive query vector sequence, and the first embedding feature includes: determining an alignment loss based on a difference between the mapped feature and the second embedded feature; Determining a codebook loss based on a first difference between the interactive query vector sequence and the first embedded feature; determining a consistency loss based on a second difference between the interaction query vector sequence and the first embedding feature; A first loss value is determined based on the alignment loss, the codebook loss, and the consistency loss.

7. The method according to any one of claims 1 to 4, characterized in that: The feature interaction processing is implemented by a pre-trained query converter in the molecular marker, and before iteratively executing multiple training batches on the molecular marker, the method further includes: A plurality of training batches are performed on the query converter, wherein the following processes are performed in each training batch of the query converter: Acquire a text feature sample of a molecule description text sample through the query converter; Obtaining a sample of encoded features of a sample of molecular graph data; Performing a first attention processing on the text feature sample and a second query vector sequence of a current training batch of the query converter by the query converter to obtain a first attention feature sample, wherein when the current training batch of the query converter is the first training batch, the second query vector of the current training batch of the query converter is randomly initialized; Performing a second attention processing on the first attention feature sample and the encoding feature sample by the query converter to obtain a second attention feature sample; Performing feed-forward processing on the second attention feature sample by the query converter to obtain an interactive query sequence sample; A plurality of training tasks are performed based on the interactive query sequence samples and the text feature samples to obtain the pre-trained query converter.

8. The method according to claim 7, characterized in that The multiple training tasks include a molecular text comparison learning training task, a molecular text matching training task, and a molecular text generation training task. The multiple training tasks are performed based on the interactive query sample and the text feature sample to obtain the pre-trained query converter, including: Determining a first sub-loss value corresponding to the molecular text contrastive learning training task based on the interactive query sample and the text feature sample; Obtaining a text embedding feature sample of the molecule description text sample; Determining a second sub-loss value corresponding to the molecular text matching training task based on the interactive query sample, the encoding feature sample and the text embedding feature sample; Determining a third sub-loss value corresponding to the molecular text generation training task based on the text embedding feature sample and the encoding feature sample; Based on the first sub-loss value, the second sub-loss value and the third sub-loss value, the query converter and the second query vector sequence of the current training batch are updated, wherein the updated query converter and the second query vector sequence are used for the next batch of training of the query converter, and the second query vector sequence updated in the last training batch is used as the first query vector sequence of the first training batch of the molecular marker.

9. The method according to claim 1, characterized in that: The step of executing the training task of the molecular data processing model based on the trained molecular marker comprises: Based on the molecular marker sequence, the molecular representation data and the prompt, a first autoregressive training task is performed on the pre-trained language understanding model to obtain a first language understanding model for generating the molecular description text; Based on the molecular marker sequence, the prompt and the molecular description text, a second autoregressive training task is performed on the first language understanding model to obtain a second language understanding model for generating the molecular description text and for generating the molecular marker sequence, wherein the second language understanding model serves as the molecular data processing model.

10. The method according to claim 9, characterized in that The method of performing a first autoregressive training task on a pre-trained language understanding model based on the molecular marker sequence, the molecular representation data and the prompt to obtain a first language understanding model for generating the molecular description text includes: Connecting the molecular tag sequence, the embedded representation of the molecular representation data, the embedded representation of the prompt, and the embedded representation of the molecular description text to obtain a first multimodal sequence; Masking the embedded representation of the molecular description text in the first multimodal sequence to obtain a first input sequence; Performing a first prediction process on the embedded representation of the masked molecular description text in the first input sequence through the pre-trained language understanding model to obtain a first prediction result; Determining a first prediction loss value based on the first prediction result and the embedded representation of the molecule description text; Update the pre-trained language understanding model based on the first prediction loss value to obtain the first language understanding model for generating the molecular description text; The step of performing a second autoregressive training task on the first language understanding model based on the molecular marker sequence, the prompt and the molecular description text to obtain a second language understanding model for generating the molecular description text and for generating the molecular marker sequence includes: Connecting the molecular marker sequence, the embedded representation of the prompt, and the embedded representation of the molecular description text to obtain a second multimodal sequence; Masking the molecular marker sequence in the second multimodal sequence to obtain a second input sequence; Performing a second prediction process on the molecular marker sequence using the first language understanding model to obtain a second prediction result; Determine a second prediction loss value based on the second prediction result and the molecular marker sequence; The first language understanding model is updated based on the second prediction loss value to obtain the second language understanding model used to generate the molecular description text and the molecular marker sequence.

11. The method according to claim 1, characterized in that: The step of executing the training task of the molecular data processing model based on the trained molecular marker comprises: Based on the molecular marker sequence, the molecular representation data and the prompt, a first autoregressive training task is performed on the pre-trained language understanding model to obtain a first molecular data processing model for generating the molecular description text; Based on the molecular marker sequence, the prompt and the molecular description text, a second autoregressive training task is performed on the pre-trained language understanding model to obtain a second molecular data processing model for generating the molecular marker sequence.

12. The method according to any one of claims 9 to 11, characterized in that: After executing the training task of the molecular data processing model based on the trained molecular marker, the method further comprises: Acquire training data for a specific task related to molecular processing, wherein the training data for the specific task includes a molecular data sample and annotated data sample; By using the molecular data processing model, the molecular data sample is processed for prediction of a specific task result to obtain prediction data; Determining a specific task loss value based on the labeled data sample and the predicted data; The molecular data processing model is updated based on the specific task loss value to obtain a molecular data processing model for the specific task.

13. A molecular data processing method, characterized in that: The method comprises: Obtaining molecular data to be processed and prompts to be processed; In response to the molecular data to be processed representing a molecular structure, performing molecular marker generation processing on the molecular data to be processed by a molecular marker to obtain a molecular marker sequence of the molecular data to be processed; Through a molecular data processing model, molecular description text prediction processing is performed based on the molecular marker sequence of the molecular data to be processed and the embedded representation of the prompt to be processed to obtain the molecular description text of the molecular data to be processed, wherein the molecular marker and the molecular data processing model are trained by the method described in any one of claims 1 to 11.

14. A molecular data processing method, characterized in that: The method comprises: Obtaining molecular data to be processed and prompts to be processed; In response to the fact that the molecular data to be processed is text data, molecular marker prediction processing is performed through the molecular data processing model based on the embedded representation of the molecular data to be processed and the embedded representation of the prompt to be processed to obtain a molecular marker sequence of the molecular data to be processed, wherein the molecular marker and the molecular data processing model are trained by the method described in any one of claims 1 to 11.

15. A training device for a molecular data processing model, characterized in that: The device comprises: A first training module is used to iteratively execute multiple training batches for the molecular marker, wherein the following processing is performed in each training batch: feature interaction processing is performed on the molecular description text, the molecular graph data and the first query vector sequence of the current training batch by the molecular marker to obtain an interactive query sequence, querying the codebook of the current training batch based on the interactive query sequence to obtain a molecular marker sequence, determining a first loss value based on the interactive query sequence and the molecular marker sequence, and updating the molecular marker, the first query vector sequence and the codebook based on the first loss value, wherein the updated molecular marker, the first query vector sequence and the codebook are used for the next training batch of the molecular marker; The second training module is used to perform the training task of the molecular data processing model based on the molecular markers after the multiple training batches.

16. A molecular data processing device, characterized in that: The device comprises: A data processing module, used to obtain molecular data to be processed and prompts to be processed; The data processing module is further used for performing molecular marker generation processing on the molecular data to be processed by a molecular marker to obtain a molecular marker sequence of the molecular data to be processed in response to the molecular data to be processed representing a molecular structure; A prediction module is used to perform molecular description text prediction processing based on the molecular marker sequence of the molecular data to be processed and the embedded representation of the prompt to be processed through a molecular data processing model to obtain the molecular description text of the molecular data to be processed, wherein the molecular marker and the molecular data processing model are trained by the method described in any one of claims 1 to 11.

17. A molecular data processing device, characterized in that: The device comprises: A data processing module, used to obtain molecular data to be processed and prompts to be processed; A prediction module is used for, in response to the molecular data to be processed being text data, performing molecular marker prediction processing based on the embedded representation of the molecular data to be processed and the embedded representation of the prompt to be processed through the molecular data processing model to obtain a molecular marker sequence of the molecular data to be processed, wherein the molecular marker and the molecular data processing model are trained by the method described in any one of claims 1 to 11.

18. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions or computer programs; A processor, for implementing the training method of the molecular data processing model described in any one of claims 1 to 12, or implementing the molecular data processing method described in claim 13 or claim 14, when executing the computer executable instructions or computer program stored in the memory.

19. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the training method of the molecular data processing model described in any one of claims 1 to 12 is implemented, or the molecular data processing method described in claim 13 or claim 14 is implemented.

20. A computer program product comprising computer executable instructions or a computer program, characterized in that When the computer executable instructions or computer program are executed by a processor, the training method of the molecular data processing model described in any one of claims 1 to 12 is implemented, or the molecular data processing method described in claim 13 or claim 14 is implemented.

Images