Molecular data processing model training method, molecular data processing method, device, equipment, storage medium and program product

Abstract

The application provides a training method and device of a molecular data processing model, a molecular data processing method and device, a storage medium and a program product. The method comprises: iteratively performing a plurality of training batches on a molecular marker; in each training batch, performing the following processing: performing feature interaction processing on a molecular description text, molecular graph data and a first query vector sequence of the current training batch by the molecular marker to obtain an interaction query sequence, querying an encoding base of the current training batch based on the interaction query sequence to obtain a molecular marker sequence, determining a first loss value based on the interaction query sequence and the molecular marker sequence, updating the molecular marker, the first query vector sequence and the encoding base based on the first loss value; and performing a training task of the molecular data processing model based on the molecular marker after the plurality of training batches. Through the application, the consistency of multi-modal data representation can be improved, and the generalization ability of the molecular data processing model can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a training method for a molecular data processing model, a molecular data processing method, an apparatus, a device, a storage medium, and a program product. Background Technology

[0002] Molecular data processing methods in related technologies integrate molecular information by fusing molecular data from different modalities (such as molecular graph data and text data), such as by using multi-head attention mechanisms or cross-attention mechanisms. Although this method can process multimodal data of molecules and text, the processing of multimodal data still has limitations because the data of different modalities have different structures and characteristics, which affects the performance of molecular data processing models. Summary of the Invention

[0003] This application provides a training method for a molecular data processing model, a molecular data processing method, an apparatus, a device, a storage medium, and a program product, which can improve the consistency of multimodal data representation and enhance the generalization ability of the molecular data processing model.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a method for training a molecular data processing model, wherein the molecular data processing model is trained with the aid of molecular markers, and the method includes:

[0006] Multiple training batches are iteratively performed on the molecular marker, with the following processing performed in each training batch:

[0007] The molecular marker is used to perform feature interaction processing on the molecular description text, molecular graph data, and the first query vector sequence of the current training batch to obtain an interactive query sequence.

[0008] Based on the interactive query sequence, the current training batch's encoding is queried to obtain the molecular marker sequence.

[0009] A first loss value is determined based on the interactive query sequence and the molecular marker sequence.

[0010] The molecular marker, the first query vector sequence, and the codebook are updated based on the first loss value, wherein the updated molecular marker, the first query vector sequence, and the codebook are used for the next training batch of the molecular marker;

[0011] Based on the molecular markers obtained from the multiple training batches, the training task of the molecular data processing model is performed.

[0012] This application provides a molecular data processing method, the method comprising:

[0013] Obtain the molecular data to be processed and the processing prompts;

[0014] In response to the molecular structure represented by the molecular data to be processed, molecular marker generation processing is performed on the molecular data to be processed by a molecular marker generator to obtain the molecular marker sequence of the molecular data to be processed.

[0015] Molecular descriptive text prediction processing is performed on the molecular label sequence of the molecular data to be processed and the embedded representation of the prompt to be processed by the molecular data processing model to obtain the molecular descriptive text of the molecular data to be processed. The molecular labeler and the molecular data processing model are trained by the molecular data processing model training method provided in the embodiments of this application.

[0016] This application provides a molecular data processing method, the method comprising:

[0017] Obtain the molecular data to be processed and the processing prompts;

[0018] In response to the fact that the molecular data to be processed is text data, molecular marker prediction processing is performed based on the embedded representation of the molecular data to be processed and the embedded representation of the prompt to be processed through the molecular data processing model to obtain the molecular marker sequence of the molecular data to be processed. The molecular marker and the molecular data processing model are trained by the training method of the molecular data processing model provided in the embodiments of this application.

[0019] This application provides a training device for a molecular data processing model, comprising:

[0020] The first training module is used to iteratively execute multiple training batches on the molecular marker, wherein the following processes are performed in each training batch: the molecular marker performs feature interaction processing on the molecular description text, molecular graph data and the first query vector sequence of the current training batch to obtain an interaction query sequence; the codebook of the current training batch is queried based on the interaction query sequence to obtain a molecular marker sequence; a first loss value is determined based on the interaction query sequence and the molecular marker sequence; the molecular marker, the first query vector sequence and the codebook are updated based on the first loss value, wherein the updated molecular marker, the first query vector sequence and the codebook are used for the next training batch of the molecular marker;

[0021] The second training module is used to perform the training task of the molecular data processing model based on the molecular markers after the multiple training batches.

[0022] This application provides a molecular data processing apparatus, including:

[0023] The data processing module is used to acquire molecular data to be processed and processing prompts;

[0024] The data processing module is further configured to, in response to the molecular structure represented by the molecular data to be processed, perform molecular marker generation processing on the molecular data to be processed through a molecular marker generator to obtain the molecular marker sequence of the molecular data to be processed;

[0025] The prediction module is used to perform molecular description text prediction processing based on the molecular marker sequence of the molecular data to be processed and the embedded representation of the prompt to be processed through a molecular data processing model, so as to obtain the molecular description text of the molecular data to be processed. The molecular marker and the molecular data processing model are trained by the training method of the molecular data processing model provided in the embodiments of this application.

[0026] This application provides a molecular data processing apparatus, including:

[0027] The data processing module is used to acquire molecular data to be processed and processing prompts;

[0028] The prediction module is used to, in response to the fact that the molecular data to be processed is text data, perform molecular marker prediction processing based on the embedding representation of the molecular data to be processed and the embedding representation of the prompt to be processed through the molecular data processing model to obtain the molecular marker sequence of the molecular data to be processed. The molecular marker and the molecular data processing model are trained by the training method of the molecular data processing model provided in the embodiments of this application.

[0029] This application provides an electronic device, the electronic device comprising:

[0030] Memory is used to store executable instructions or computer programs.

[0031] The processor is configured to execute computer-executable instructions or computer programs stored in the memory to implement the training method for the molecular data processing model provided in the embodiments of this application, or to implement the molecular data processing method provided in the embodiments of this application.

[0032] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the training method for the molecular data processing model provided in this application, or implements the molecular data processing method provided in this application.

[0033] This application provides a computer program product, including a computer program or computer-executable instructions. When the computer program or computer-executable instructions are executed by a processor, they implement the training method for the molecular data processing model provided in this application, or implement the molecular data processing method provided in this application.

[0034] The embodiments of this application have the following beneficial effects:

[0035] By performing multiple training batches on the molecular tagger, feature interaction processing is applied to molecular graph data and molecular descriptive text, continuously updating the first query vector sequence and the encoding text. The interactive query sequence obtained from feature interaction processing maps the molecular graph data to the molecular tag sequence, aligning the molecular representation sequence and the molecular descriptive text representation. This improves the consistency of multimodal data representation. This unified representation eliminates encoding differences between different modalities, enabling the trained molecular data processing model to process and fuse multimodal information more naturally, accurately capturing and aligning the deep relationships between molecules and text. Specifically, through feature interaction processing of molecular graph data and molecular descriptive text, the model can learn the correlation between the molecular graph data and molecular descriptive text. This interaction can reveal the implicit relationship between molecular structure and its descriptive text, thereby improving the molecular data processing model's understanding of molecular information and achieving the beneficial effect of enhancing the generalization ability of the molecular data processing model. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the molecular data processing system architecture provided in the embodiments of this application;

[0037] Figure 2A This is a first structural diagram of the server provided in an embodiment of this application;

[0038] Figure 2B This is a schematic diagram of the second structure of the server provided in an embodiment of this application;

[0039] Figure 2C This is a schematic diagram of the third structure of the server provided in the embodiments of this application;

[0040] Figure 3A This is a schematic diagram of the first process of the training method for the molecular data processing model provided in the embodiments of this application;

[0041] Figure 3B This is a schematic diagram of the second process of the training method for the molecular data processing model provided in the embodiments of this application;

[0042] Figure 3C This is a schematic diagram of the third process of the training method for the molecular data processing model provided in the embodiments of this application;

[0043] Figure 3D This is a schematic diagram of the fourth process of the training method for the molecular data processing model provided in the embodiments of this application;

[0044] Figure 3E This is a schematic diagram of the fifth step of the training method for the molecular data processing model provided in the embodiments of this application;

[0045] Figure 3F This is a schematic diagram of the sixth step in the training method of the molecular data processing model provided in the embodiments of this application;

[0046] Figure 3G This is a schematic diagram of the seventh process of the training method for the molecular data processing model provided in the embodiments of this application;

[0047] Figure 3H This is a schematic diagram of the eighth step in the training method of the molecular data processing model provided in the embodiments of this application;

[0048] Figure 3I This is a schematic diagram of the ninth step in the training method of the molecular data processing model provided in the embodiments of this application;

[0049] Figure 3J This is a schematic diagram of the tenth step of the training method for the molecular data processing model provided in the embodiments of this application;

[0050] Figure 3K This is a schematic diagram of the eleventh step of the training method for the molecular data processing model provided in the embodiments of this application;

[0051] Figure 3L This is a schematic diagram of the twelfth step of the training method for the molecular data processing model provided in the embodiments of this application;

[0052] Figure 3M This is a schematic diagram of the thirteenth step of the training method for the molecular data processing model provided in the embodiments of this application;

[0053] Figure 4 This is a schematic diagram of the first process of the molecular data processing method provided in the embodiments of this application;

[0054] Figure 5 This is a schematic diagram of the second process of the molecular data processing method provided in the embodiments of this application;

[0055] Figure 6 This is a flowchart illustrating the application of a molecular data processing method in a molecular property prediction scenario provided in this application embodiment;

[0056] Figure 7A This is a first schematic diagram of an optional structure of the molecular data processing model provided in the embodiments of this application;

[0057] Figure 7BThis is a second schematic diagram of an optional structure of the molecular data processing model provided in the embodiments of this application;

[0058] Figure 7C This is a schematic diagram illustrating the training principle of the first language understanding model provided in the embodiments of this application;

[0059] Figure 7D This is a schematic diagram illustrating the training principle of the second language understanding model as described in the embodiments of this application.

[0060] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0062] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0063] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0064] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0065] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0066] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0067] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0068] 1) In response to, used to indicate the conditions or states on which the operation performed depends. When the conditions or states on which it depends are met, one or more operations performed may be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0069] 2) Molecular graph data is a graphical data format used to represent and store molecular structures. In this format, a molecule is represented as a graph, where nodes represent atoms and edges represent chemical bonds. Molecular graph data can capture the topological structure and chemical properties of molecules and is a commonly used data format in cheminformatics and molecular modeling. The following are some key components of molecular graph data:

[0070] Nodes (atoms): In a molecular diagram, each node represents an atom. Nodes typically indicate the type of atom (e.g., carbon, hydrogen, oxygen) and its properties (e.g., charge, whether it is part of a ring, etc.).

[0071] Edges (chemical bonds): The edges between nodes represent chemical bonds between atoms. Edges can be directed (representing single, double, or triple bonds) or undirected (representing any type of chemical bond). Edges may also contain properties of the bond, such as the bond strength or type (single, double, triple, or aromatic).

[0072] Graph structure: The structure of a molecular graph is defined by the set of nodes and edges. It can be a simple small molecular graph containing only a few atoms and bonds, or a complex large molecular graph containing hundreds of atoms and bonds.

[0073] Additional information: In addition to the basic graph structure, molecular graph data may also include other information, such as the three-dimensional coordinates of the molecule, molecular weight, molecular formula, functional group information, etc.

[0074] 3) Molecular representation data, also known as the Simplified Molecular Input Line Entry System (SMILES) and SMILES strings, is a simplified linear text representation used to describe the structure of chemical substances. SMILES strings are widely used in cheminformatics and drug design.

[0075] A SMILES string (numerator representing data) includes the following basic elements:

[0076] Atoms: Atoms are represented using symbols from the periodic table, such as "C" for carbon atoms and "H" for hydrogen atoms.

[0077] Chemical bonds: Specific symbols are used to represent the way atoms are connected. Single bonds do not require a special symbol, double bonds are represented by an equal sign "=", and triple bonds are represented by a hash symbol "#".

[0078] Branching: Parentheses are used to indicate branching structures in a molecule.

[0079] Ring structure: Use numbers to mark the opening and closing positions of the ring structure.

[0080] Electric charge: The charge of an atom is represented by the plus sign "+" and the minus sign "-".

[0081] Here are some examples of SMILES strings:

[0082] Water: O or H2O.

[0083] Ethanol: CO or COC.

[0084] Benzene: c1cccccc1.

[0085] Acetone: CC(=O)C.

[0086] In the benzene SMILES string c1ccCcc1 above, c represents a carbon atom, and the number 1 indicates the closure of the ring, meaning that the first and last carbon atoms form a ring.

[0087] The advantage of SMILES strings lies in their conciseness and readability, allowing chemists to quickly record and share molecular structures. Furthermore, SMILES strings can be parsed by computer programs to generate two-dimensional and three-dimensional molecular structures, as well as to perform various chemical calculations and analyses.

[0088] 4) Molecular description text is a textual form that uses natural language or semi-structured language to describe the structure of chemical molecules, including the composition of the molecule, the type of atoms, the type of chemical bonds, the geometric structure of the molecule, and other possible chemical properties.

[0089] Here are some key elements included in a molecular description text:

[0090] Atomic information: describes the atoms present in a molecule, including the types of atoms (such as carbon, hydrogen, oxygen, etc.) and the number of atoms.

[0091] Chemical bond information: describes how atoms are connected, including single bonds, double bonds, triple bonds, etc.

[0092] Molecular structure: describes the spatial structure of a molecule, which may include the molecule's stereochemical configuration.

[0093] Functional group: refers to a specific functional group in a molecule, such as hydroxyl, ketone, amino, etc.

[0094] Molecular properties: may include the physical and chemical properties of molecules, such as molecular weight, boiling point, melting point, etc.

[0095] Chemical reaction characteristics: Describes the role and changes of molecules in chemical reactions.

[0096] For example, the molecular description of ethanol can be represented as follows: Ethanol (Ethyl Alcohol), commonly known as alcohol or spirits, is a type of alcohol compound with the molecular formula C2H6O. It consists of two carbon atoms connected by a single bond. The first carbon atom is bonded to three hydrogen atoms, and the second carbon atom is bonded to two hydrogen atoms and a hydroxyl group (-OH). Ethanol has excellent combustibility and is a commonly used fuel, solvent, and disinfectant, and is widely used in organic synthesis.

[0097] 5) Transformer: A temporal model based on self-attention. In the encoder part, it can effectively encode temporal information, and its processing capability for temporal information is far superior to Long Short-Term Memory (LSTM) networks, while also being faster. It is widely used in natural language processing, computer vision, machine translation, speech recognition, and other fields.

[0098] 6) Large Language Models (LLMs) are large-scale language models designed to understand and generate human language. They are trained on massive amounts of text data and can perform a wide range of tasks, including text summarization, translation, sentiment analysis, and more. LLMs are characterized by their enormous scale, containing billions of parameters that help them learn complex patterns in language data. They are typically based on deep learning architectures. LLMs refer to deep learning models trained on massive amounts of text data, containing billions or even more parameters. They can be used to generate natural language text and understand its meaning. Through training, the models learn the statistical regularities and semantic relationships of language to build a vast language knowledge base, thereby simulating human language understanding and generation capabilities. LLMs have the following characteristics:

[0099] Learning ability: Through training with massive amounts of text data, large language models can learn rich language knowledge and expressions, including grammar, semantics and common expression habits.

[0100] Pattern recognition: Large language models can identify common text patterns and semantic relationships, such as co-occurrence relationships between words, logical structure of sentences, and semantic roles.

[0101] Contextual understanding: Large language models can capture contextual information in text, understand the influence of previous text on subsequent text, and generate corresponding responses based on the context.

[0102] Generative capabilities: Large language models can generate relevant natural language text based on input information, including answering questions, generating articles, and engaging in dialogue.

[0103] Resolving ambiguity: Despite the existence of polysemy and ambiguity in language, large language models resolve ambiguity through contextual information and linguistic rules, providing more accurate and appropriate text generation or understanding.

[0104] Large language models have a wide range of applications, including intelligent customer service, intelligent question answering, natural language generation, advertising recommendation, and games. They can improve the efficiency and accuracy of human-computer interaction and enhance the user experience.

[0105] 7) The Querying Transformer (Q-Former) is a query-based transformer model that interacts with molecular features extracted by a frozen encoder using learnable query vectors. These queries are crucial for extracting relevant information from the molecular features. Q-Former consists of a Molecule Transformer and a Text Transformer, which share a self-attention layer. The Molecule Transformer incorporates a cross-attention layer between its self-attention and feedforward layers, while the Text Transformer's architecture can be built upon the Pre-training of Deep Bidirectional Transformers for Language Understanding (BERT). Q-Former employs a cross-attention mechanism where the query vectors selectively focus on different aspects of the molecular features, enabling the model to capture the key details necessary to understand molecular properties and generate textual descriptions.

[0106] 8) A molecular tokenizer is a model used to segment molecular map data into smaller, meaningful units or "tags." These molecular tokens help identify and predict the structure and function of molecules when processing molecular data. Specifically, a molecular tokenizer can be viewed as a model that transforms molecular map data into a format that machine learning models can process.

[0107] 9) Molecular data processing model is a large language model based on the Transformer architecture, specifically designed for processing molecular data. Through pre-training, it can learn the complex distribution of molecular data and can be used for molecular property prediction, drug design and other molecular-related tasks.

[0108] 10) A graph neural network (GNN) is a type of neural network that takes graph data containing points, edges and their attributes as input and makes predictions.

[0109] Molecular data processing methods in related technologies use independent molecular and text embeddings, and then integrate this information through some form of fusion mechanism, such as multi-head attention or cross-attention, by calculating the similarity between different modalities and mapping them to a shared representation space. While this approach can handle multimodal data of molecules and text, it still has limitations in terms of data representation consistency and the complexity of the generation task.

[0110] The technical solutions of related technologies mainly include the following aspects:

[0111] 1) Multimodal alignment models: These models obtain embedding representations of different modalities by independently encoding molecules and text, and then use alignment mechanisms to achieve cross-modal interaction.

[0112] 2) Self-supervised learning methods: Using self-supervised learning techniques such as contrastive learning, general multimodal representations are learned by designing specific pre-training tasks, such as molecular text contrastive learning training tasks.

[0113] 3) Molecular generation models: Traditional molecular generation models usually use graph neural networks (GNNs) or generative models based on SMILES representation to generate new molecular structures, but these models usually only focus on the molecule itself and do not involve multimodal generation.

[0114] Multimodal alignment models are a class of models designed to map data from different modalities into a shared representation space. These models typically include independent encoders for processing different modalities, such as text and molecules. For text data, pre-trained language models are commonly used encoders, trained on large amounts of natural language text to generate rich text embeddings. For molecular data, models typically use graph neural networks (GNNs) or sequence models based on SMILES representations to generate molecular embeddings. GNNs utilize the graph structure to encode the atoms and bonds of molecules as nodes and edges, while SMILES-based models represent molecules as character sequences and generate corresponding embeddings. In multimodal alignment models, after being encoded by their respective encoders, data from different modalities undergo some form of alignment mechanism to achieve cross-modal interaction. This alignment mechanism typically employs multi-head attention or cross-attention mechanisms, calculating the similarity between different modalities and mapping them to a shared representation space. Within this shared space, the embeddings of different modalities can be used for various cross-modal tasks, such as cross-modal retrieval and cross-modal generation.

[0115] Self-supervised learning methods are a class of approaches that utilize unlabeled data to train models, particularly suitable for multimodal tasks. Unlike supervised learning, which relies on large amounts of labeled data, self-supervised learning enables models to automatically learn effective feature representations from raw data by designing specific pre-training tasks. In multimodal learning, self-supervised learning methods typically design synthetic tasks or signals, allowing the model to autonomously acquire supervisory signals from the input data. These tasks can include various forms such as contrastive learning, masked language modeling, and generative adversarial networks. Contrastive learning is a common self-supervised learning method that trains the model by comparing positive and negative sample pairs to learn general multimodal representations. Specifically, contrastive learning generates pairs of samples from the dataset, where the two data instances in a pair either belong to the same category or different categories. In a molecular text contrastive learning training task, a positive sample pair can be a molecule and its corresponding text description, while a negative sample pair is a combination of a molecule and unrelated text. During the learning process, the model attempts to make the representations of positive sample pairs as close as possible to those of negative sample pairs and as far away from them as possible. This approach forces the model to learn which features are important factors in distinguishing the similarity between modalities, thereby gradually building effective representations without explicit labels.

[0116] Molecular generative models are a class of machine learning models focused on generating and optimizing chemical molecular structures. They utilize data-driven methods to learn how to generate molecules with specific physical and chemical properties from combinations of atoms and bonds. Molecular generative models have important applications in drug discovery, materials design, and chemical synthesis. Traditional molecular generative models typically rely on graph neural networks (GNNs) or sequence models based on SMILES representations for modeling. GNN models learn a global representation of a molecular graph by recursively passing and aggregating information between nodes (atoms) and edges (chemical bonds). This process is analogous to diffusing information within a molecular graph, ensuring that the final representation of each node includes structural information about that node and its neighborhood. This approach captures the complex topological features of molecular structures, making it well-suited for generating new molecules. By learning the distribution of molecular graphs on training data, GNNs can construct new molecular structures that meet specific requirements during the generation phase. Furthermore, GNN models can integrate attribute information from nodes and edges, such as the types of atoms and bonds, to more accurately reflect the chemical properties of the molecule. On the other hand, sequence models based on SMILES representations use a simplified linear representation of molecules, transforming molecules into string sequences. This representation flattens the two-dimensional structure of a molecule into a one-dimensional sequence, preserving the molecule's composition and connectivity information. Using sequence generation models, such as recurrent neural networks (RNNs) or Transformers, the model can predict the next character during character-by-character generation, thus constructing a complete SMILES sequence. The advantages of this method lie in its simplicity of generation and ease of integration with language models. By training on a large amount of SMILES data, the model can capture the grammatical rules and common structural patterns of chemical molecules, thereby generating or synthesizing reasonable and novel molecules.

[0117] The limitations of the relevant technologies are mainly reflected in the following aspects:

[0118] 1) Insufficient alignment and fusion of multimodal data: Existing multimodal alignment models typically employ independent encoders and alignment mechanisms to process data from different modalities. This approach can lead to information loss and insufficient alignment, especially when dealing with data of varying complexity and scale. Representations generated by independent encoders may not fully capture the deep relationships between different modalities, which limits the model's performance in multimodal tasks.

[0119] 2) The complexity of designing self-supervised learning tasks: Self-supervised learning methods rely on carefully designed pre-training tasks, the selection and design of which require extensive experimentation and experience. Inappropriate task design can lead to poor generalization ability in the representations learned by the model. Furthermore, training self-supervised learning models typically requires significant computational resources, especially when dealing with large-scale multimodal data.

[0120] 3) Limitations of Molecular Generation Models: Traditional molecular generation models, such as GNNs and SMILES-based sequence models, while performing well in generating molecules, have limitations in handling interactions with other modalities such as text. These models typically focus only on the structural generation of molecules, neglecting the fusion of multimodal information, making it difficult to achieve joint generation and understanding of multimodal data. Furthermore, these models may struggle to generate molecules with specific properties, especially when faced with complex design requirements.

[0121] This application provides a training method for a molecular data processing model, a molecular data processing method, an apparatus, a device, a computer-readable storage medium, and a computer program product, which can improve the consistency of multimodal data representation and enhance the generalization ability of the molecular data processing model. The following describes exemplary applications of the electronic devices provided in this application. The electronic devices provided in this application can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, smartphones, smart speakers, smartwatches, smart TVs, and vehicle terminals, or they can be implemented as servers.

[0122] See Figure 1 , Figure 1 This is a schematic diagram of the molecular data processing system architecture provided in the embodiments of this application. Figure 1 The system involves server 100, terminal device 200, and network 300. Terminal device 200 is connected to server 100 through network 300, which can be a wide area network (WAN), a local area network (LAN), or a combination of both.

[0123] In some embodiments, the present application embodiments can be implemented collaboratively by a server and a terminal device. For example, server 100 obtains a molecular data processing model using the training method of the molecular data processing model provided in the present application embodiments, and terminal device 200 sends the molecular data to be processed and the processing prompt to server 100. Server 100 obtains the molecular description text or the molecular marker sequence of the molecular data to be processed using the molecular data processing method provided in the present application embodiments, and sends the molecular description text or the molecular marker sequence of the molecular data to be processed (which can be used to reconstruct the molecular structure) as the processing result to terminal device 200.

[0124] Here, server 100 can be a single server. In this case, the training method and molecular data processing method of the molecular data processing model provided in this application embodiment can be implemented by the same server. Server 100 can also be a cluster of servers. In the case that server 100 is a server cluster, the training method and molecular data processing method of the molecular data processing model provided in this application embodiment can be implemented by different servers. This application embodiment does not impose any limitations on this.

[0125] In other embodiments, the embodiments of this application can be implemented independently by a terminal device. The terminal device 200 sends a request to the server 100. The server 100 receives the request and sends a molecular data processing model for performing the molecular data processing method provided in the embodiments of this application to the terminal device 200. The terminal device 200 receives the molecular data processing model sent by the server and downloads it locally. It then obtains the molecular description text or molecular marker sequence of the molecular data to be processed through the molecular data processing model.

[0126] Taking the use of a server for training molecular data processing models as an example, see [link to relevant documentation]. Figure 2A , Figure 2A This is a first structural diagram of the server provided in an embodiment of this application. Figure 2A The server 100-1 shown includes at least one processor 110-1, memory 130-1, and at least one network interface 120-1. The various components in server 100-1 are coupled together via a bus system 140-1. It is understood that the bus system 140-1 is used to implement communication between these components. In addition to a data bus, the bus system 140-1 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2A The general designated all buses as Bus System 140-1.

[0127] The processor 110-1 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0128] The memory 130-1 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 130-1 may optionally include one or more storage devices physically located away from the processor 110-1.

[0129] The memory 130-1 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 130-1 described in this application embodiment is intended to include any suitable type of memory.

[0130] In some embodiments, memory 130-1 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0131] Operating system 131-1 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0132] The network communication module 132-1 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 120-1, such as Bluetooth, WiFi, and Universal Serial Bus (USB).

[0133] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2A A training device 133 for a molecular data processing model stored in memory 130-1 is shown. This device can be software in the form of programs and plugins, and includes the following software modules: a first training module 1331 and a second training module 1332. These modules are logically linked and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0134] Taking the use of a server for molecular data processing as an example, see [link to relevant documentation]. Figure 2B , Figure 2B This is a schematic diagram of the second structure of the server provided in an embodiment of this application. Figure 2B The server 100-2 shown includes at least one processor 110-2, memory 130-2, and at least one network interface 120-2. The various components in server 100-2 are coupled together via a bus system 140-2. It is understood that the bus system 140-2 is used to implement communication between these components. In addition to a data bus, the bus system 140-2 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2BAll buses are labeled as Bus System 140-2. For detailed descriptions of Processor 110-2 and Memory 130-2, please refer to the above text; they will not be repeated here.

[0135] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2B A molecular data processing device 134 stored in memory 130-2 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a data processing module 1341 and a prediction module 1342. These modules are logically linked and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0136] Taking the use of a server for molecular data processing as an example, see [link to relevant documentation]. Figure 2C , Figure 2C This is a schematic diagram of the third structure of the server provided in the embodiments of this application. Figure 2C The server 100-3 shown includes at least one processor 110-3, memory 130-3, and at least one network interface 120-3. The various components in server 100-3 are coupled together via a bus system 140-3. It is understood that the bus system 140-3 is used to implement communication between these components. In addition to a data bus, the bus system 140-3 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2C All buses are labeled as Bus System 140-3. For detailed explanations of Processor 110-3 and Memory 130-3, please refer to the above text; they will not be repeated here.

[0137] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2C A molecular data processing device 135 stored in memory 130-3 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a data processing module 1351 and a prediction module 1352. These modules are logically linked and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0138] In some embodiments, the terminal device or server can implement the training method or molecular data processing method of the molecular data processing model provided in this application embodiment by running various computer-executable instructions or computer programs. For example, computer-executable instructions can be microprogram-level commands, machine instructions, or software instructions. Computer programs can be native programs or software modules in an operating system; they can be native applications (APPs), i.e., programs that need to be installed in the operating system to run, such as chemical database APPs or scientific research collaboration APPs; or they can be applets that can be embedded in any APP, i.e., programs that only need to be downloaded to a browser environment to run. In summary, the above-mentioned computer-executable instructions can be any form of instruction, and the above-mentioned computer programs can be any form of application, module, or plugin.

[0139] In other embodiments, the apparatus provided in this application can be implemented in hardware. As an example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the training method or molecular data processing method of the molecular data processing model provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0140] The following will describe the training method of the molecular data processing model provided in this application embodiment, with the server as the execution subject, using exemplary applications and implementations of the server provided in the embodiments of this application. See also Figure 3A , Figure 3A This is a schematic diagram of the first flowchart of the training method for the molecular data processing model provided in the embodiments of this application, which will be combined with Figure 3A The steps shown are explained.

[0141] In step 101, multiple training batches are iteratively performed on the molecular marker, wherein the following steps 1011 to 1014 are performed in each training batch.

[0142] See Figure 7A , Figure 7AThis is a first schematic diagram of an optional structure of the molecular data processing model provided in the embodiments of this application. Figure 7A An optional structure of the molecular marker is shown. Molecular map data is input into a pre-trained molecular encoder to obtain encoded features. The encoded features, a learnable first query vector sequence, and molecular description text are input into a query converter for feature interaction processing (corresponding to step 1011 below) to obtain an interactive query sequence. By querying the nearest neighbors of the interactive query sequence in the learnable encoded text, a molecular marker sequence is obtained.

[0143] In step 1011, the molecular description text, molecular graph data and the first query vector sequence of the current training batch are processed by a molecular tagger to obtain an interactive query sequence.

[0144] In some embodiments, see Figure 3B , Figure 3A Step 1011 shown can be implemented through steps 10111 to 10113, as explained in detail below.

[0145] In step 10111, the text features of the molecular description text and the first query vector sequence are subjected to first attention processing to obtain the first attention features.

[0146] In some embodiments, see Figure 3C , Figure 3B Step 10111 shown can be achieved through steps 201 to 204, which are explained in detail below.

[0147] In step 201, a first attention score matrix is ​​determined based on the textual features of the molecular description text and the first query vector sequence.

[0148] In some embodiments, the text features of the molecular description text can be obtained by: performing word segmentation on the molecular description text to obtain input units; performing token embeddings on the input units to obtain token embedding features; and performing self-attention encoding on the token embedding features to obtain the text features of the molecular description text.

[0149] For example, punctuation marks such as spaces, periods, and commas can be used as segmentation markers to segment molecular description text, that is, to divide the molecular description text into input units. Third-party segmentation tools can also be used to segment molecular description text, such as Jieba, NLTK, and SpaCy. These tools can achieve accurate Chinese and English word segmentation through algorithms and language models.

[0150] For example, word embedding models (such as Word2Vec, GloVe, etc.) can be used to encode the input units, representing each input unit as a fixed-length vector, and then forming word embedding features (sequences) in order.

[0151] For example, word embedding features can be used as query vectors (Q), key vectors (K), and value vectors (V). Attention scores are obtained by calculating the dot product of Q and K. Next, the attention scores are normalized, for example by applying a normalization function (such as the softmax function), to transform the attention scores into a probability distribution, which is then used as attention weights. The attention weights are used to weight the word embedding features (corresponding to V above) to generate new feature representations. These new feature representations can be linearly transformed through a linear layer to obtain a rich representation that includes the correlation between different positions in the word embedding features, i.e., text features that describe the text.

[0152] In some embodiments, the first attention score matrix is ​​determined based on the text features of the molecular description text and the first query vector sequence. This can be achieved by using the first query vector sequence as the query vector (Query, Q) and the text features as both the key vector (Key, K) and the value vector (Value, V). The first attention score matrix is ​​obtained by calculating the dot product of Q and K.

[0153] For example, assuming Q and K contain three elements (q1, q2, q3) and (k1, k2, k3) respectively, the resulting first attention score matrix can be represented as: {(q1k1: 0.1, q1k2: 0.2, q1k3: 0.7), (q2k1: 0.4, q2k2: 0.3, q2k3: 0.3), (q3k1: 0.2, q3k2: 0.5, q3k3: 0.3)}.

[0154] In step 202, the mask matrix of the first attention score matrix is ​​obtained.

[0155] In some embodiments, a causal mask can be used as the mask matrix for the first attention score matrix.

[0156] For example, the mask matrix can be represented as: {(1, -∞, -∞), (1, 1, -∞), (1, 1, 1)}.

[0157] In step 203, the first attention score matrix is ​​masked based on the mask matrix, and the resulting mask score matrix is ​​normalized to obtain the first attention weight matrix.

[0158] In some embodiments, the mask matrix is ​​multiplied by the corresponding elements of the first attention score matrix to obtain the mask score matrix.

[0159] Following the example above, the mask fraction matrix can be represented as: {(q1k1: 0.1, q1k2: -∞, qlk3: -∞), (q2k1: 0.4, q2k2: 0.3, q2k3: -∞), (q3k1: 0.2, q3k2: 0.5, q3k3: 0.3)}.

[0160] Following the example above, the mask score matrix can be normalized using the softmax function, and the resulting first attention weight matrix can be represented as: {(qlk1: 1.00, q1k2: 0.00, qlk3: 0.00), (q2k1: 0.57, q2k2: 0.43, q2k3: 0.00), (q3k1: 0.31, q3k2: 0.41, q3k3: 0.28)}.

[0161] In step 204, the text features are weighted and summed using the first attention weight matrix to obtain the first attention features.

[0162] Continuing with the previous example, the text features (v1, v2, v3) are weighted and summed using the first attention weight matrix. For q1: qlk1*v1 = v1; for q2: q2k1*v1 + q2k2*v2 = 0.57vl + 0.43v2; for q3: q3kl*v1 + q3k2*v2 + q3k3*v3 = 0.31v1 + 0.41v2 + 0.28v3. The first attention feature can be represented as: (v1, 0.57vl + 0.43v2, 0.31v1 + 0.41v2 + 0.28v3). It can be seen that q1 only focuses on k1, q2 mainly focuses on k1, slightly focuses on k2, and doesn't focus on k3, while q3 focuses on k1, k2, and k3. This ensures that when the model predicts the output at any given time point, it only sees the values ​​k1, k2, and k3. Information at or before a given point in time, but not information about the future, aligns with the causal relationships of events in the real world. This helps the model (molecular marker) make more reasonable and accurate predictions. For example, if q2 could pay attention to k3, it might receive hints about future information, violating the order constraints in natural language. By restricting q2 from paying attention to k3, the model avoids prematurely acquiring information it shouldn't know. When the model learns to make predictions with limited information, it helps it learn to extract and utilize key features, improving its generalization ability. At the same time, this method of limiting the scope of attention reduces computation because the model does not need to calculate attention weights between each query vector (corresponding to q1, q2, and q3 above) and all key vectors (corresponding to k1, k2, and k3 above).

[0163] For example, in steps 201 to 204, the text features of the molecular description text and the first query vector sequence can be processed by a causal attention mask to obtain the first attention features. The causal attention mask ensures that when the model predicts the output at any point in time, it can only see the information at that point in time and before, and cannot access the information in the future. This is consistent with the causal laws of events in the real world and helps the model make more reasonable and accurate predictions.

[0164] See also Figure 3B In step 10112, the first attention feature and the encoded features of the molecular graph data are subjected to second attention processing to obtain the second attention feature.

[0165] In some embodiments, the structural information of a molecule is represented as a graph (molecular graph data), using... It means that among them It is a set of atoms, where ε represents the edge between atoms. It is the number of atoms. The molecular graph data is processed by feature encoding through a pre-trained molecular encoder to obtain the encoded features of the molecular graph data.

[0166] Taking a pre-trained molecular encoder as an example of a multi-modal molecule structure-text model for text-based retrieval and editing (MoleculeSTM), the encoded features of molecular graph data can be obtained as follows: First, a feature vector is assigned to each atom and edge in the molecular graph data. Atom features can include atom type, charge, whether it is part of a ring, etc.; edge features can include chemical bond type, bond orientation, etc. Next, an embedding layer is used to convert the pre-processed atom and edge features into fixed-size embedding vectors. These vectors will be used as input in subsequent Transformer layers. Finally, a graph attention matrix is ​​constructed based on the embedding representations of nodes and edges. This matrix represents the degree of correlation between the atoms in the molecule. Through the Transformer encoder (which includes multiple self-attention layers and feedforward neural network layers), the attention score of each node is calculated in the self-attention layer to determine the relationship between nodes and update the node representation accordingly. In the feedforward network layer, the model further processes the representation of each node to extract more complex features. After each self-attention layer and feedforward network layer, residual connections and layer normalization are performed. After passing through multiple Transformer encoder layers, the embedding vector of each node is updated. These vectors can be used as the final feature representation of the node, thus obtaining the encoded features of the molecular graph data.

[0167] For example, the coding features of molecular map data can be represented as follows: Here, F represents the number of features, which means that each atom has F different features, such as atomic charge, atomic radius, electronegativity, etc. Each atomic feature contains context-aware structural information to reflect the relationship and interaction between the atom and its surrounding atoms, such as the distance, angle and torsion angle with other atoms.

[0168] In some embodiments, a second attention feature is obtained by performing a second attention process on the first attention feature and the encoded features of the molecular graph data through a cross-attention mechanism.

[0169] For example, the first attention feature is used as the query vector (Q), and the encoded features of the molecular graph data are used as the key vector (K) and value vector (V). The attention score is obtained by calculating the dot product of Q and K. Next, the attention score is normalized, for example by applying a normalization function (such as the softmax function), so that the attention score becomes a probability distribution, which is used as the attention weight. The attention weight is used to weight the encoded features (V) of the molecular graph data to generate a new feature representation. The new feature representation can be linearly transformed through a linear layer to obtain a rich representation that includes the correlation between different positions in the encoded features of the molecular graph data, i.e., the second attention feature.

[0170] In step 10113, the second attention features are processed by feedforward to obtain the interactive query sequence.

[0171] In some embodiments, the second attention features are processed by a feedforward network layer (such as a fully connected layer or a linear layer) to obtain an interactive query sequence.

[0172] For example, the second attention features are linearly mapped using the weight matrix and bias vector of the feedforward network layer to obtain linear features. The linear features are then non-linearly mapped using an activation function (such as sigmoid) to obtain the interactive query sequence.

[0173] For example, an interactive query sequence can be represented as Where M represents the number of queries, which refers to the number of interactive query vectors in the interactive query sequence, that is, Z includes M interactive query vectors z. i d represents the dimension of each interactive query vector, that is, the number of elements in the interactive query vector. This dimension determines the amount of information and complexity that each interactive query vector can carry.

[0174] See also Figure 3A In step 1012, the current training batch's encoding is queried based on the interactive query sequence to obtain the molecular marker sequence.

[0175] In some embodiments, the interactive query vector sequence includes multiple interactive query vectors, see [link to relevant documentation]. Figure 3D , Figure 3A Step 1012 shown can be implemented through steps 10121 to 10122, which are explained in detail below.

[0176] In step 10121, from the encoding book of the current training batch, the nearest encoding vector of each interactive query vector is queried, where the nearest encoding vector is the encoding vector with the smallest distance to the interactive query vector, and the index of each nearest encoding vector is queried.

[0177] In some embodiments, the codebook includes an encoding vector and a corresponding index, derived from the codebook of the current training batch. In the context of the size of the encoded text, the nearest encoded vector for each interactive query vector can be represented by formula (1):

[0178] s i =argmin j{1，...，K} ||z i -c j ||2 (1)

[0179] Where i = 1, 2, ..., M, is the interactive query vector z. i Quantized to a vector c in the encoding. j s i It can be viewed as an index, i.e., s i Is it that ||z i -c j The smallest j in ||2, for example, given an interactive query vector z1 and a codebook (c1, c2, c3), calculate ||z. i -c j ||2, Assuming the distance between z1 and c2 is minimized, i.e., c2 is the nearest encoded vector of z1, then s i =2.

[0180] In step 10122, the indices of each most recent coding vector are combined to obtain a molecular marker sequence.

[0181] In some embodiments, the index combination of the most recent encoded vector of each interactive query vector in the interactive query vector sequence is used as a molecular marker sequence.

[0182] Following the example above, the molecular representation sequence can be represented as:

[0183] See also Figure 3A In step 1013, a first loss value is determined based on the interactive query sequence and the molecular marker sequence.

[0184] In some embodiments, the molecular marker sequence includes multiple molecular markers, see [link to documentation]. Figure 3E , Figure 3A Step 1013 shown can be implemented through steps 10131 to 10134, which will be explained in detail below.

[0185] In step 10131, the encoding vector indexed by each molecular marker is queried from the encoding book of the current training batch, and the encoding vector corresponding to each molecular marker is combined into the first embedding feature.

[0186] In some embodiments, molecular marker sequences As an index, query the encoding vectors in the encoding book of the current training batch, and combine the encoding vectors corresponding to each molecular marker into the first embedding feature.

[0187] Following the example above, the first embedding feature can be represented as

[0188] In step 10132, the first embedded feature is subjected to feature mapping processing to obtain the mapped feature.

[0189] In some embodiments, a multilayer perceptron (MLP) can be used as an adapter to perform feature mapping on the first embedded features to obtain mapped features.

[0190] For example, via MLP (corresponding to Figure 7A The adapter in the multilayer perceptron performs feature mapping on the first embedded feature to obtain the mapped feature. This can be achieved by linearly mapping the second attention feature using the weight matrix and bias vector of the fully connected layer in the multilayer perceptron to obtain the linear feature, and then performing non-linear mapping on the linear feature using an activation function (such as sigmoid) to obtain the mapped feature.

[0191] For example, it can be done through Figure 7A The pre-trained molecular representation data decoder shown in the figure performs molecular reconstruction based on mapping features to obtain a reconstructed molecular map.

[0192] Following the example above, the mapping feature can be represented as X R =ψ(C), where ψ represents the multilayer perceptron and C represents the first embedded feature.

[0193] In step 10133, the molecular representation data is subjected to embedding encoding processing to obtain the second embedding feature.

[0194] In some embodiments, the molecular representation data is embedded and encoded using a pre-trained molecular representation data encoder (SMILES Encoder) to obtain a second embedded feature.

[0195] For example, a pre-trained molecular representation data encoder (corresponding to...) Figure 7ATaking the molecular representation data shown in the example of a pre-trained Transformer for Computational Chemistry (Chemformer) as an example, the molecular representation data (SMILES strings) undergoes embedding encoding to obtain the second embedding feature. This can be achieved as follows: SMILES string preprocessing: First, each character in the SMILES string (including atoms, chemical bonds, branch points, etc.) is converted into a corresponding category label or atom type. Special symbols in the SMILES string (such as brackets for branch points) are converted into special tokens that can represent molecular structures. A vocabulary is constructed containing all possible SMILES characters and tokens. This vocabulary is used to convert the SMILES string into integer indices. The SMILES string is then converted into a sequence of integers using the vocabulary, with each integer corresponding to a specific character or token in the vocabulary. To preserve positional information in the sequence, positional encoding is applied to the sequence. The positional encoding is generated by sine and cosine functions, or it can be a learned fixed vector. The integer sequence and positional encoding are passed as input to the encoder of ChemFormer. The encoder of ChemFormer encodes the integer sequence and positional encoding to obtain the second embedding feature.

[0196] For example, ChemFormer's encoder consists of multiple identical layers, each performing the following operations: Through a self-attention mechanism, the model assigns different attention weights to each element in the sequence to capture its meaning based on context. After self-attention, each element undergoes further feature transformation via a feedforward network. Residual connections and layer normalization are performed after each sub-layer (self-attention and feedforward network) to improve model stability and performance. After all encoder layers, each input token (corresponding to an element in the SMILES string) has a feature vector representation containing contextual information about that token. Combining the vector representations of each token yields a second embedding feature.

[0197] In step 10134, a first loss value is determined based on the mapping features, the second embedding features, the interactive query vector sequence, and the first embedding features.

[0198] In some embodiments, see Figure 3F , Figure 3E Step 10134 shown can be implemented through steps 301 to 304, which will be explained in detail below.

[0199] In step 301, the alignment loss is determined based on the difference between the mapping feature and the second embedding feature.

[0200] In some embodiments, the alignment loss is determined based on the difference between the mapped features and the second embedded features, and can be expressed by formula (2):

[0201]

[0202] Among them, X S X represents the second embedding feature. R This indicates the mapping characteristics.

[0203] In step 302, the encoding loss is determined based on the first difference between the interactive query vector sequence and the first embedded feature.

[0204] In some embodiments, the encoding loss is determined based on the first difference between the interactive query vector sequence and the first embedded feature, and can be expressed by formula (3):

[0205]

[0206] Where M represents the number of interactive query vectors in the interactive query vector sequence (i.e., the number of queries), and sg[·] represents the stopping gradient operator. The molecular marker in the first embedding feature is s i The encoded vector, z i This represents the i-th interactive query vector in the sequence of interactive query vectors.

[0207] In step 303, the consistency loss is determined based on the second difference between the interactive query vector sequence and the first embedded feature.

[0208] In some embodiments, the consistency loss is determined based on the second difference between the interactive query vector sequence and the first embedded feature, and can be expressed by formula (4):

[0209]

[0210] Wherein, β represents a hyperparameter, and in this embodiment of the application, β is set to 0.25.

[0211] In step 304, a first loss value is determined based on alignment loss, codebook loss, and consistency loss.

[0212] Following the example above, based on the alignment loss, codebook loss, and consistency loss, the first loss value is determined and can be expressed by formula (5):

[0213]

[0214] The first term represents the alignment loss (corresponding to formula (2)), the second term is the codebook loss aimed at updating the codebook embedding (corresponding to formula (3)), and the third term is the commitment loss that encourages the interactive query vector sequence to remain consistent with the embedding representation (first embedding feature) of the selected codebook (corresponding to formula (4)).

[0215] See also Figure 3A In step 1014, the molecular marker, the first query vector sequence, and the codebook are updated based on the first loss value, wherein the updated molecular marker, the first query vector sequence, and the codebook are used for the next training batch of the molecular marker.

[0216] In some embodiments, gradient information is obtained through a first loss value, and the molecular marker, the first query vector sequence, and the codebook are updated based on the gradient information to obtain the updated molecular marker, the first query vector sequence, and the codebook.

[0217] For example, the gradient information of the first loss value with respect to each parameter of the molecular marker, the first query vector sequence, and the codebook is obtained through the backpropagation algorithm. The parameters of the molecular marker, the first query vector sequence, and the codebook are updated using the obtained gradient information according to the gradient descent optimization algorithm (such as batch gradient descent, stochastic gradient descent, etc.). The above process is repeated until a certain number of iterations are reached or the molecular marker, the first query vector sequence, and the codebook converge, thereby obtaining the trained molecular marker, the first query vector sequence, and the codebook.

[0218] For example, since the vector quantization process from the interactive query sequence to the molecular marker sequence is non-differentiable, a pass-through estimation method is used to copy the gradient information from the molecular marker sequence to the interactive query sequence for gradient update.

[0219] Through steps 1011 to 1014, a molecular tokenizer is used to unify different modalities of data, such as molecules and text, into discrete token representations. This unified representation eliminates the encoding differences between different modalities, enabling the model to process and fuse multimodal information more naturally. It can more accurately capture and align the deep relationships between molecules and text, significantly improving the performance of molecular data processing in multimodal tasks.

[0220] In some embodiments, see Figure 3G Feature interaction processing is implemented through a query converter pre-trained in the molecular marker. Before iterating through multiple training batches of the molecular marker, multiple training batches can also be performed on the query converter. In each training batch of the query converter, the following steps 401 to 406 are performed, which are explained in detail below.

[0221] For example, see Figure 7B , Figure 7B This is a second schematic diagram of an optional structure of the molecular data processing model provided in the embodiments of this application. Figure 7B A schematic diagram of an optional structure of the query converter is shown. Each training batch of the query converter performs three training tasks. The query converter and the second query vector sequence obtained after multiple training batches will be used for the processing in step 1011.

[0222] In step 401, the text feature samples of the molecular description text sample are obtained by query converter.

[0223] In some embodiments, the text feature samples of the molecular description text sample can be obtained by: performing word segmentation on the molecular description text sample to obtain input units; performing token embeddings on the input units to obtain token embedding features; and performing self-attention encoding on the token embedding features to obtain the text feature samples of the molecular description text sample.

[0224] For example, punctuation marks such as spaces, periods, and commas can be used as segmentation markers to segment molecular description text samples, that is, to divide the molecular description text samples into input units. Alternatively, third-party segmentation tools such as Jieba, NLTK, and SpaCy can be used to segment molecular description text samples. These tools can achieve accurate Chinese and English word segmentation through algorithms and language models.

[0225] For example, word embedding models (such as Word2Vec, GloVe, etc.) can be used to encode the input units, representing each input unit as a fixed-length vector, and then forming word embedding features (sequences) in order.

[0226] For example, word embedding features can be used as query vectors (Q), key vectors (K), and value vectors (V). Attention scores are obtained by calculating the dot product of Q and K. Next, the attention scores are normalized, for example, by applying a normalization function (such as the softmax function), transforming the attention scores into a probability distribution, which serves as the attention weights. These attention weights are then used to weight the word embedding features (corresponding to V above) to generate new feature representations (corresponding to...). Figure 7B By performing self-attention calculation on the molecular description text (Yang Ben), the new feature representation can be linearly transformed through a linear layer to obtain a rich representation that includes the correlation of different positions in the word embedding features, i.e., the text feature sample of the molecular description text sample.

[0227] In step 402, the encoded feature samples of the molecular map data samples are obtained.

[0228] In some embodiments, a pre-trained molecular encoder is used to perform feature encoding on the molecular graph data samples to obtain encoded feature samples of the molecular graph data samples.

[0229] For example, a pre-trained molecular encoder (corresponding to) Figure 7B The molecular encoder shown here, using the Multi-modal molecule structure-text model for text-based retrieval and editing (MoleculeSTM) as an example, can obtain the encoded features of molecular graph data samples as follows: First, a feature vector is assigned to each atom and edge in the molecular graph data sample. Atom features can include atom type, charge, whether it is part of a ring, etc.; edge features can include chemical bond type, bond orientation, etc. Next, an embedding layer is used to convert the preprocessed atom and edge features into fixed-size embedding vectors. These vectors will be used as input in subsequent Transformer layers. Finally, a graph attention matrix is ​​constructed based on the embedding representations of nodes and edges. This matrix represents the degree of correlation between the atoms in the molecule. Through the Transformer encoder (which includes multiple self-attention layers and feedforward neural network layers), the attention score of each node is calculated in the self-attention layer to determine the relationship between nodes and update the node representation accordingly. In the feedforward network layer, the model further processes the representation of each node to extract more complex features. After each self-attention layer and feedforward network layer, residual connections and layer normalization are performed. After passing through multiple Transformer encoder layers, the embedding vector of each node is updated. These vectors can be used as the final feature representation of the node, thus obtaining the encoded feature samples of the molecular graph data sample.

[0230] In step 403, the text feature samples and the second query vector sequence of the current training batch of the query converter are subjected to first attention processing by the query converter to obtain the first attention feature samples. When the current training batch of the query converter is the first training batch, the second query vector of the current training batch of the query converter is randomly initialized.

[0231] In some embodiments, a first attention score matrix sample is determined based on text feature samples and a second query vector sequence (see the description of step 201 above); a mask matrix sample of the first attention score matrix sample is obtained (see the description of step 202 above); the first attention score matrix sample is masked based on the mask matrix sample, and the obtained mask score matrix sample is normalized to obtain a first attention weight matrix sample (see the description of step 203 above); the text feature samples are weighted and summed using the first attention weight matrix sample to obtain a first attention feature sample (see the description of step 204 above).

[0232] In step 404, the first attention feature sample and the encoded feature sample are subjected to second attention processing by a query converter to obtain the second attention feature sample.

[0233] In some embodiments, a second attention process is performed on the first attention feature samples and the encoded feature samples using a cross-attention mechanism (corresponding to...). Figure 7B The cross-attention shown in the figure is used to obtain the second attention feature sample. For specific implementation methods, please refer to the description in step 10112 above, which will not be repeated here.

[0234] In step 405, the second attention feature samples are processed by a query converter to obtain interactive query sequence samples.

[0235] In some embodiments, the second attention feature samples are processed by a feedforward network layer (such as a fully connected layer or a linear layer) to obtain interactive query sequence samples. For specific implementation methods, please refer to the description of step 10113 above, which will not be repeated here.

[0236] In step 406, multiple training tasks are performed based on interactive query sequence samples and text feature samples to obtain a pre-trained query converter.

[0237] In some embodiments, the multiple training tasks include a molecular text contrast learning training task, a molecular text matching training task, and a molecular text generation training task, see [link to documentation]. Figure 3H , Figure 3G Step 406 shown can be implemented through steps 4061 to 4065, as explained in detail below.

[0238] In step 4061, the first sub-loss value corresponding to the molecular text contrast learning training task is determined based on the interactive query samples and text feature samples.

[0239] In some embodiments, the first sub-loss value corresponding to the Molecule-Text Contrastive Learning (MTC) training task is determined based on interactive query samples and text feature samples, and can be expressed by formula (6):

[0240]

[0241] Where B represents the batch size, and τ represents the temperature parameter (hyperparameter). yi and yi refer to the i-th interactive query sample and the text feature sample in the same training batch, respectively.

[0242] Here, the Molecular Text Contrast Learning (MTC) training task aims to align molecular and text features by maximizing the information between them. This is achieved by maximizing the molecular-text similarity of positive sample pairs and the similarity of negative sample pairs. This is done using the last interactive query vector sample z from the interactive query samples. M As a representation of the query, the output interactive query samples (a sequence of multiple interactive query vector samples) are causally related, meaning that the last interactive query vector sample contains global information from the interactive query samples.

[0243] In step 4062, text embedding feature samples of molecular description text samples are obtained.

[0244] In some embodiments, the text embedding feature samples of molecular description text samples can be obtained by: performing word segmentation on the molecular description text samples to obtain input units; and performing token embeddings on the input units to obtain text embedding feature samples for each word.

[0245] In step 4063, the second sub-loss value corresponding to the molecular text matching training task is determined based on the interactive query sample, the encoded feature sample, and the text embedding feature sample.

[0246] In some embodiments, the second sub-loss value corresponding to the Molecule-Text Matching (MTM) training task is determined based on interactive query samples, encoded feature samples, and text embedding feature samples, and can be expressed by formula (7):

[0247]

[0248] Where φ represents a binary classifier, X i and t iLet z refer to the encoded feature sample and the text embedding feature sample of the i-th input in a training batch, respectively. The last interactive query vector sample z in the interactive query samples... M As a representation of a query.

[0249] Specifically, due to the interactive query sequence samples Information about the molecular modality and the text modality was captured through first attention processing and second attention processing, respectively. This was then utilized using the last interactive query vector sample z from the interactive query samples. M As input to a binary classifier, this classifier predicts whether a given molecule-text pair matches.

[0250] In step 4064, the third sub-loss value corresponding to the molecular text generation training task is determined based on the text embedding feature samples and the encoding feature samples.

[0251] In some embodiments, a third sub-loss value is determined based on text embedding feature samples and encoded feature samples for the Molecule-grounded Text Generation (MTG) training task. Specifically, MTG refers to generating molecular text description samples given encoded feature sample input. In this task, since only text output is required, a causal mask is not applied to the second query vector sequence, but a causal mask is applied to the text embedding feature samples. That is, each text token (text embedding feature sample of each word) is allowed to focus on the preceding text tokens and all second query vector sequences, but not on subsequent text tokens (corresponding to...). Figure 7B Different masking strategies are used for different training tasks. The Language Modeling (LM) loss function is used to model the performance of a given input X. i Generate t under the condition i This can be expressed as formula (8):

[0252]

[0253] in, Representative text embedding feature sample t i The j-th text tag in the text.

[0254] In step 4065, based on the first sub-loss value, the second sub-loss value, and the third sub-loss value, the query converter and the second query vector sequence of the current training batch are updated. The updated query converter and the second query vector sequence are used for the training of the next batch of the query converter. The second query vector sequence obtained from the last training batch is used as the first query vector sequence of the first training batch of the molecular marker.

[0255] In some embodiments, the total loss value used to train the query converter is determined based on the first sub-loss value, the second sub-loss value, and the third sub-loss value, and can be expressed by formula (9):

[0256] L Q-Former =L MTC +L MTM +L MTG (9)

[0257] In some embodiments, gradient information is obtained through the total loss value, and the query converter and the second query vector sequence of the current training batch are updated according to the gradient information to obtain the updated query converter and the second query vector sequence.

[0258] For example, the gradient information of the total loss value with respect to each parameter of the query converter and the second query vector sequence is obtained through the backpropagation algorithm. The parameters of the query converter and the second query vector sequence are updated using the obtained gradient information according to the gradient descent optimization algorithm (such as batch gradient descent, stochastic gradient descent, etc.). The above process is repeated until a certain number of iterations are reached or the query converter and the second query vector sequence converge, thereby obtaining the trained query converter and the second query vector sequence.

[0259] See also Figure 3A In step 102, the training task of the molecular data processing model is performed based on the molecular markers after multiple training batches.

[0260] In some embodiments, see Figure 3I , Figure 3A Step 102 shown can be implemented through the following steps 501 to 502, which are explained in detail below.

[0261] In step 501, based on molecular marker sequences, molecular representation data, and prompts, the pre-trained language understanding model is subjected to a first autoregressive training task to obtain a first language understanding model for generating molecular description text.

[0262] In some embodiments, see Figure 3J , Figure 3A Step 501 shown can be implemented through steps 5011 to 5015, as explained in detail below.

[0263] In step 5011, the molecular marker sequence, the embedded representation of molecular representation data, the embedded representation of the prompt, and the embedded representation of molecular description text are concatenated to obtain the first multimodal sequence.

[0264] In some embodiments, the molecular marker sequence, the embedded representation of molecular representation data, the embedded representation of the prompt, and the embedded representation of the molecular description text are concatenated together to obtain a first multimodal sequence.

[0265] For example, molecular marker sequences can be obtained through Figure 7C The molecular markers shown (with parameters unchanged during training in step 501) are obtained by processing the molecular map data. The embedded representations of the molecular representation data and the embedded representations of the prompts can be obtained through... Figure 7C The text tokenizer mapping shown in the figure is obtained.

[0266] For example, to facilitate the representation of multimodal sequences, a molecular vocabulary is constructed. The molecular vocabulary maintains the order of the codebook. For example, if c1 in the codebook is the representation of a molecule, then the molecule vocabulary... This corresponds to the discrete label. Furthermore... It also includes special markers, such as boundary indicators like [MOL] and [ / MOL], to mark the beginning and end of the molecular sequence. Next, the text vocabulary will be... With molecular vocabulary Synthesized into a unified molecular-text vocabulary This facilitates joint learning of molecules and text under a unified next-label prediction objective. As the vocabulary expands, the corresponding embedding and prediction layers also need to be expanded; the newly introduced parameters are initialized using a random initialization method.

[0267] In step 5012, the embedded representation of the molecular description text in the first multimodal sequence is masked to obtain the first input sequence.

[0268] In some embodiments, the embedding representation of molecular descriptive text in the first multimodal sequence is masked, that is, the embedding representation of molecular descriptive text is used as the generation target of the first autoregressive training task to obtain the first input sequence.

[0269] In step 5013, the embedded representation of the masked molecular description text in the first input sequence is subjected to a first prediction process using a pre-trained language understanding model to obtain a first prediction result.

[0270] In some embodiments, a pre-trained language understanding model (e.g., a large language model) is used to predict the embedding representation of the molecular description text of the mask, thereby recovering the masked portion (corresponding to...). Figure 7CThe process can involve using the encoder and decoder of a language understanding model to generate a first prediction. For example, the language understanding model uses its encoder to process the unmasked portion of the first input sequence and computes the hidden state at each position. For the embedded representation of the masked molecular description text, the language understanding model predicts the original molecular description text based on the encoded contextual information (the hidden state at each position). The language understanding model then applies a softmax function to the decoder to generate a probability distribution (representing the probability value of each word in the predicted molecular description text) as the first prediction, or outputs the predicted molecular description text as the first prediction (corresponding to...). Figure 7C (The molecular description text output by the first language understanding model is shown in the figure).

[0271] In step 5014, a first prediction loss value is determined based on the first prediction result and the embedded representation of the molecular description text.

[0272] In some embodiments, the first prediction loss value is determined based on the first prediction result and the embedded representation of the molecular description text, and can be expressed by formula (10):

[0273]

[0274] in, Representative dataset (using multimodal data), Let θ represent the set of exponents that generate the target (an embedded representation of the molecular descriptive text), and let θ represent the parameters of the language understanding model for each element u in the sequence. i The model will compute the values ​​of all preceding elements u1, ..., u2. i-1 time u i The conditional probability p is calculated, then the logarithm is taken and summed. A unified representation and training paradigm for molecules and text enhances the language understanding model's ability to understand and register molecule-text interactions.

[0275] In step 5015, the pre-trained language understanding model is updated based on the first prediction loss value to obtain the first language understanding model used to generate molecular description text.

[0276] In some embodiments, gradient information is obtained through a first prediction loss value, and the pre-trained language understanding model is updated based on the gradient information to obtain an updated first language understanding model.

[0277] For example, the gradient information of the first predicted loss value with respect to each parameter of the language understanding model is obtained through the backpropagation algorithm. The parameters of the language understanding model are updated using the obtained gradient information according to the gradient descent optimization algorithm (such as batch gradient descent, stochastic gradient descent, etc.). The above process is repeated until a certain number of iterations are reached or the language understanding model converges, thereby obtaining the first language understanding model after training.

[0278] See also Figure 3I In step 502, based on the molecular marker sequence, prompts, and molecular description text, the first language understanding model is subjected to a second autoregressive training task to obtain a second language understanding model for generating molecular description text and for generating molecular marker sequences, wherein the second language understanding model serves as a molecular data processing model.

[0279] In some embodiments, see Figure 3K , Figure 3I Step 502 shown can be implemented through steps 5021 to 5025, which will be explained in detail below.

[0280] In step 5021, the molecular marker sequence, the embedded representation of the prompt, and the embedded representation of the molecular description text are concatenated to obtain the second multimodal sequence.

[0281] For a detailed description of the implementation method, please refer to step 5011 above; it will not be repeated here.

[0282] In step 5022, the molecular marker sequence in the second multimodal sequence is masked to obtain the second input sequence.

[0283] In some embodiments, the molecular marker sequences in the first two-modal sequences are masked, that is, the molecular marker sequences are used as the generation target of the second autoregressive training task (corresponding to...). Figure 7D (The mask shown in the image) is used to obtain the second input sequence.

[0284] In step 5023, the molecular marker sequence is subjected to a second prediction process using a first language understanding model to obtain a second prediction result.

[0285] In some embodiments, the molecular marker sequence of the mask is predicted by a pre-trained language understanding model (e.g., a large language model) to recover the masked portion. This process may include using the encoder and decoder of the language understanding model to generate a second prediction result. For a specific implementation, please refer to the description of step 5013. The prediction processes are similar, but the difference lies in the generation target.

[0286] For example, the predicted molecular marker sequence (corresponding to) Figure 7DThe molecular marker sequence output by the second language understanding model shown in the figure can be used for molecular reconstruction to obtain the reconstructed molecule (corresponding to...). Figure 7D (The model reasoning shown in the figure).

[0287] In step 5024, a second prediction loss value is determined based on the second prediction result and the molecular marker sequence.

[0288] In some embodiments, the determination of the second prediction loss value based on the second prediction result and the molecular marker sequence can be found in the explanation of formula (10) above, that is, the generation target in formula (10) is converted into a molecular marker sequence.

[0289] In step 5025, the first language understanding model is updated based on the second prediction loss value to obtain a second language understanding model for generating molecular description text and for generating molecular marker sequences.

[0290] For a detailed description of the implementation method, please refer to step 5015 above; it will not be repeated here.

[0291] Through steps 501 and 502, utilizing an autoregressive training model, not only can molecular descriptive text be generated, but also molecular marker sequences. This unified generative framework supports the joint generation and understanding of multimodal information, providing a more flexible and universal generation method. This enables the generation of molecules with specific properties, along with corresponding textual descriptions, in fields such as drug discovery and materials design, thereby accelerating the discovery and development of new molecules.

[0292] In some embodiments, see Figure 3L , Figure 3A Step 102 shown can be implemented through steps 503 to 504, which are explained in detail below.

[0293] In step 503, based on molecular marker sequences, molecular representation data, and prompts, the pre-trained language understanding model is subjected to a first autoregressive training task to obtain a first molecular data processing model for generating molecular description text.

[0294] For specific implementation details, please refer to the descriptions of steps 5011 to 5015 above. The first language understanding model corresponds to the first molecular data processing model, which will not be elaborated here.

[0295] In step 504, based on the molecular marker sequence, prompts, and molecular description text, a second autoregressive training task is performed on the pre-trained language understanding model to obtain a second molecular data processing model for generating molecular marker sequences.

[0296] Here, for specific implementation details, please refer to the description of steps 5021 to 5025 above. A second autoregressive training task is performed on the basis of the pre-trained language understanding model to update the parameters of the pre-trained language understanding model and obtain a second molecular data processing model for generating molecular marker sequences.

[0297] In some embodiments, see Figure 3M After training the molecular data processing model based on the trained molecular markers, the following steps 601 to 604 can also be performed, which are explained in detail below.

[0298] In step 601, training data for a specific task related to molecular processing is acquired, wherein the training data for the specific task includes molecular data samples and labeled data samples.

[0299] In some embodiments, specific tasks related to molecular processing may include: molecular property prediction, used to predict various physical, chemical, or biological activity properties of molecules (such as solubility, activity, stability, etc.); molecular nomenclature, used to convert the structural information of a molecule into a chemical name or output a textual description of the molecule; molecular text retrieval, used to extract and retrieve information related to a specific molecule from textual data, such as searching for descriptions, properties, or related research of a molecule from scientific literature, patents, or online databases; molecular generation, used to generate the corresponding molecular structure based on the textual description of the molecule; reagent prediction, used to predict the chemical reagents that may be used in a specific reaction, which can help chemists select appropriate reagents to optimize the reaction pathway or improve the yield; forward reaction prediction, used to predict the chemical reactions that may occur under specific conditions for a given reactant, which can help chemists design new synthetic routes or optimize existing reactions; and retrosynthesis, a method of inferring possible synthetic pathways from a target molecule, which can be used in medicinal chemistry to help chemists find effective methods for synthesizing target molecules.

[0300] Taking molecular property prediction as an example for a specific task, the training data for the specific task can include molecular data samples (such as molecular diagram data samples, molecular description text samples, and molecular representation data samples) and labeled data samples (such as molecular solubility, molecular stability, etc.).

[0301] In step 602, the molecular data samples are processed using a molecular data processing model to predict the results of a specific task, thereby obtaining the predicted data.

[0302] In some embodiments, molecular data samples are processed by a molecular data processing model (second language understanding model). Based on the learned ability to understand molecular-text interaction and registration, the molecular data samples are processed to predict the results of a specific task, thereby obtaining predicted data.

[0303] Taking the prediction of molecular properties as an example for a specific task, the prediction data may include solubility (unit: mol / L or g / L), decomposition temperature, half-life, reaction rate constant, etc.

[0304] In step 603, the loss value for a specific task is determined based on the labeled data samples and the predicted data.

[0305] In some embodiments, the difference between the labeled data samples and the predicted data is obtained through a pre-set loss function (such as the cross-entropy loss function) and used as the loss value for a specific task.

[0306] In step 604, the molecular data processing model is updated based on the specific task loss value to obtain a molecular data processing model for the specific task.

[0307] In some embodiments, gradient information is obtained through a specific task loss value, and the molecular data processing model is updated based on the gradient information to obtain an updated molecular data processing model for a specific task.

[0308] For example, the gradient information of the loss value for a specific task with respect to each parameter of the molecular data processing model can be obtained through the backpropagation algorithm. The parameters of the molecular data processing model can be updated using the obtained gradient information according to the gradient descent optimization algorithm (such as batch gradient descent, stochastic gradient descent, etc.). The above process is repeated until a certain number of iterations are reached or the molecular data processing model converges, thereby obtaining the trained molecular data processing model for a specific task.

[0309] For example, low-rank adaptation (LoRA) can be used to introduce additional low-rank matrix parameters into certain network layers of a molecular data processing model. These low-rank matrix parameters are relatively few in number, so they do not significantly increase the computational and storage burden. Therefore, for a specific task, only the parameters of these low-rank matrices are trained, rather than the parameters of the entire pre-trained model, thereby improving training efficiency.

[0310] Through steps 601 to 604, the multimodal unified representation and generation capabilities of the trained molecular data processing model are utilized, making it widely applicable in various application scenarios (specific tasks). It provides effective solutions for molecular-text generation, molecular attribute prediction, and cross-modal information retrieval. Specifically, the unified architecture design of the molecular data processing model simplifies system integration and deployment, enhances the model's adaptability in different tasks and domains, and overcomes the limitations of traditional models in multimodal data processing through unified representation and generation methods. This improves the performance of the molecular data processing model in various multimodal tasks, not only enhancing its performance but also expanding the boundaries of multimodal data analysis and generation, providing new tools and methods for research and applications in related fields.

[0311] See Figure 4 , Figure 4 This is a schematic diagram of the first process of the molecular data processing method provided in the embodiments of this application.

[0312] In step 701, the molecular data to be processed and the processing prompt are obtained.

[0313] In some embodiments, molecular data to be processed can be obtained from molecular databases such as PubChem and DrugBank. These databases provide a large amount of molecular structure and related chemical information. In interactive applications, prompts to be processed can be obtained through specific queries or instructions input by the user, or predefined prompts can be obtained as prompts to be processed. These predefined prompts can be used to guide the molecular data processing model to perform specific tasks, such as generating molecular descriptions and predicting molecular properties. When processing dialogue systems or question-and-answer systems, prompts to be processed can also be extracted from the history of the dialogue to provide contextual information. The embodiments of this application do not limit the specific implementation of obtaining molecular data to be processed and prompts to be processed.

[0314] In step 702, in response to the molecular structure represented by the molecular data to be processed, molecular marker generation processing is performed on the molecular data to be processed by a molecular marker generator to obtain the molecular marker sequence of the molecular data to be processed.

[0315] In some embodiments, when the molecular data to be processed represents a molecular structure (represented as a molecular diagram), a molecular marker generator is used to generate molecular markers for the molecular data to be processed, thereby obtaining a molecular marker sequence for the molecular data to be processed.

[0316] For example, the learnable query vector sequence obtained after training (corresponding to the first query vector sequence updated after the last training batch) is subjected to a first attention process (such as causal attention) to obtain a first attention feature (see the description of step 10111 above for specific implementation details); the first attention feature and the encoding features of the molecular data to be processed are subjected to a second attention process to obtain a second attention feature (see the description of step 10112 above); the second attention feature is subjected to a feedforward process to obtain an interactive query sequence (see the description of step 10113 above); from the encoding script obtained after training, the nearest encoding vector of each interactive query vector is queried, where the nearest encoding vector is the encoding vector with the smallest distance to the interactive query vector, and the index of each nearest encoding vector is queried (see the description of step 10121 above); the indices of each nearest encoding vector are combined to obtain the molecular label sequence of the molecular data to be processed (see the description of step 10122 above).

[0317] For example, a second attention process is applied to the learnable query vector sequence obtained after training and the encoded features of the molecular data to be processed to obtain second attention features (see the explanation of step 10112 above); the second attention features are then processed by feedforward to obtain an interactive query sequence (see the explanation of step 10113 above); from the encoded data obtained after training, the nearest encoded vector of each interactive query vector is queried, where the nearest encoded vector is the encoded vector with the smallest distance to the interactive query vector, and the index of each nearest encoded vector is queried (see the explanation of step 10121 above); the indices of each nearest encoded vector are combined to obtain the molecular label sequence of the molecular data to be processed (see the explanation of step 10122 above).

[0318] In step 703, molecular description text prediction processing is performed based on the molecular marker sequence and the embedded representation of the prompt to be processed in the molecular data to be processed by the molecular data processing model to obtain the molecular description text of the molecular data to be processed. The molecular marker and the molecular data processing model are trained by the training method of the molecular data processing model provided in the embodiments of this application.

[0319] In some embodiments, molecular descriptive text prediction processing is performed on the molecular label sequence of the molecular data to be processed and the embedded representation of the prompt to be processed by the molecular data processing model (see the description of steps 501 to 502 above for the training of the molecular data processing model) to obtain the molecular descriptive text of the molecular data to be processed.

[0320] For example, a molecular data processing model can be used to predict the molecular descriptive text of the molecular data to be processed. This process may include using the encoder and decoder of the molecular data processing model to generate the prediction result. For example, the molecular data processing model uses its encoder to process the molecular tag sequence and the embedded representation of the cues to be processed in the molecular data to be processed, and calculates the hidden state at each position. The molecular data processing model predicts the molecular descriptive text of the molecular data to be processed based on the encoded contextual information (the hidden state at each position).

[0321] See Figure 5 , Figure 5 This is a schematic diagram of the second process of the molecular data processing method provided in the embodiments of this application.

[0322] In step 801, the molecular data to be processed and the processing prompt are obtained.

[0323] In some embodiments, molecular data to be processed can be extracted from text resources such as scientific literature, patents, and reports using text mining and natural language processing techniques. In interactive applications, prompts to be processed can be obtained through specific queries or instructions input by the user, or predefined prompts can be obtained as prompts to be processed. These predefined prompts can be used to guide the molecular data processing model to perform specific tasks, such as generating molecular descriptions and predicting molecular properties. When processing dialogue systems or question-and-answer systems, prompts to be processed can also be extracted from the history of the dialogue to provide contextual information. The embodiments of this application do not limit the specific implementation methods for obtaining molecular data to be processed and prompts to be processed.

[0324] In step 802, in response to the fact that the molecular data to be processed is text data, molecular marker prediction processing is performed based on the embedded representation of the molecular data to be processed and the embedded representation of the prompt to be processed through the molecular data processing model to obtain the molecular marker sequence of the molecular data to be processed. The molecular marker and the molecular data processing model are trained by the training method of the molecular data processing model provided in the embodiments of this application.

[0325] In some embodiments, molecular marker prediction processing is performed based on the embedded representation of the molecular data to be processed and the embedded representation of the prompt to be processed by a molecular data processing model (see the description of steps 501 to 502 above for training of the molecular data processing model) to obtain the molecular marker sequence of the molecular data to be processed.

[0326] For example, a molecular data processing model can be used to predict the molecular marker sequence of the molecular data to be processed. This process may include using the encoder and decoder of the molecular data processing model to generate the prediction result. For example, the molecular data processing model uses its encoder to process the embedded representation of the molecular data to be processed and the embedded representation of the cue to be processed, and calculates the hidden state at each position. The molecular data processing model predicts the molecular marker sequence of the molecular data to be processed based on the encoded context information (the hidden state at each position).

[0327] It should be noted that the embodiments of this application can be applied to various molecular data processing scenarios, such as molecular property prediction, molecular naming, molecular text retrieval, molecular generation, reagent prediction, and forward reaction prediction. The following will describe an exemplary application of the embodiments of this application in the molecular property prediction scenario. See [link to documentation]. Figure 6 , Figure 6 This is a flowchart illustrating the application of molecular data processing methods in the molecular property prediction scenario provided in this application embodiment, which will be combined with... Figure 6 The steps shown are explained.

[0328] In step 901, molecular data for the properties to be predicted are obtained.

[0329] In some embodiments, molecular data for the properties to be predicted can be obtained from molecular databases such as PubChem and DrugBank by prompts entered by the user. These databases provide a large amount of molecular structure and related chemical information.

[0330] For example, the user input prompt could be "I want to know the solubility of this molecule." In the interactive system, the system further obtains molecular data on the property to be predicted based on the above. For instance, the system displays "Which molecule are you referring to? Please provide its compound identifier (CID) or the string SMILES" on the interactive interface, and the user provides the prompt "CID 987654321".

[0331] In step 902, molecular property prediction processing is performed on the molecular data of the property to be predicted using a molecular data processing model for a specific task to obtain the predicted property. The molecular data processing model for the specific task is trained using the training method of the molecular data processing model provided in the embodiments of this application.

[0332] In some embodiments, a task-specific molecular data processing model (the specific training process can be found in the description of steps 601 to 604 above) is used to perform molecular property prediction processing based on the given molecular data with properties to be predicted, and outputs a number [0, 1] as the prediction of the strength of the property based on the attribute to be predicted. For example, "0" represents poor stability and "1" represents strong stability.

[0333] The following description continues to illustrate the exemplary structure of the training device 133 for the molecular data processing model provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2A As shown, the software modules in the training device 133 for the molecular data processing model stored in memory 130-1 may include:

[0334] The first training module 1331 is used to iteratively execute multiple training batches on the molecular marker, wherein the following processes are performed in each training batch: the molecular marker performs feature interaction processing on the molecular description text, molecular graph data and the first query vector sequence of the current training batch to obtain an interaction query sequence; the codebook of the current training batch is queried based on the interaction query sequence to obtain a molecular marker sequence; a first loss value is determined based on the interaction query sequence and the molecular marker sequence; the molecular marker, the first query vector sequence and the codebook are updated based on the first loss value, wherein the updated molecular marker, the first query vector sequence and the codebook are used for the next training batch of the molecular marker.

[0335] The second training module 1332 is used to perform the training task of the molecular data processing model based on the molecular markers after the multiple training batches.

[0336] In some embodiments, the first training model 1331 is further configured to perform a first attention process on the text features of the molecular description text and the first query vector sequence to obtain a first attention feature; perform a second attention process on the first attention feature and the encoding features of the molecular graph data to obtain a second attention feature; and perform feedforward processing on the second attention feature to obtain the interactive query sequence.

[0337] In some embodiments, the first training model 1331 is further configured to: determine a first attention score matrix based on the text features of the molecular description text and the first query vector sequence; obtain a mask matrix of the first attention score matrix; perform masking processing on the first attention score matrix based on the mask matrix; normalize the obtained mask score matrix to obtain a first attention weight matrix; and perform weighted summation processing on the text features through the first attention weight matrix to obtain the first attention features.

[0338] In some embodiments, the interactive query vector sequence includes multiple interactive query vectors, and the first training model 1331 is further configured to query the nearest encoding vector of each interactive query vector from the encoding of the current training batch, wherein the nearest encoding vector is the encoding vector with the smallest distance to the interactive query vector, and query the index of each nearest encoding vector; and combine the indices of each nearest encoding vector to obtain the molecular marker sequence.

[0339] In some embodiments, the molecular marker sequence includes multiple molecular markers, and the first training model 1331 is further configured to query the encoding vector indexed by each molecular marker from the encoding of the current training batch, combine the encoding vector corresponding to each molecular marker into a first embedding feature; perform feature mapping processing on the first embedding feature to obtain a mapped feature; perform embedding encoding processing on the molecular representation data to obtain a second embedding feature; and determine a first loss value based on the mapped feature, the second embedding feature, the interactive query vector sequence, and the first embedding feature.

[0340] In some embodiments, the first training model 1331 is further configured to determine an alignment loss based on the difference between the mapping feature and the second embedding feature; determine a codebook loss based on a first difference between the interactive query vector sequence and the first embedding feature; determine a consistency loss based on a second difference between the interactive query vector sequence and the first embedding feature; and determine a first loss value based on the alignment loss, the codebook loss, and the consistency loss.

[0341] In some embodiments, the feature interaction processing is implemented through a pre-trained query converter in the molecular marker. The first training model 1331 is further configured to perform multiple training batches on the query converter, wherein the following processing is performed in each training batch of the query converter: obtaining text feature samples of molecular description text samples through the query converter; obtaining encoded feature samples of molecular graph data samples; performing a first attention processing on the text feature samples and the second query vector sequence of the current training batch of the query converter through the query converter to obtain a first attention feature sample, wherein when the current training batch of the query converter is the first training batch, the second query vector of the current training batch of the query converter is randomly initialized; performing a second attention processing on the first attention feature sample and the encoded feature sample through the query converter to obtain a second attention feature sample; performing feedforward processing on the second attention feature sample through the query converter to obtain an interactive query sequence sample; and performing multiple training tasks based on the interactive query sequence sample and the text feature sample to obtain the pre-trained query converter.

[0342] In some embodiments, the plurality of training tasks include a molecular text contrast learning training task, a molecular text matching training task, and a molecular text generation training task. The first training model 1331 is further configured to: determine a first sub-loss value corresponding to the molecular text contrast learning training task based on the interactive query sample and the text feature sample; obtain text embedding feature samples of the molecular description text sample; determine a second sub-loss value corresponding to the molecular text matching training task based on the interactive query sample, the encoded feature sample, and the text embedding feature sample; determine a third sub-loss value corresponding to the molecular text generation training task based on the text embedding feature sample and the encoded feature sample; and update the query converter and the second query vector sequence of the current training batch based on the first sub-loss value, the second sub-loss value, and the third sub-loss value. The updated query converter and the second query vector sequence are used for the training of the next batch of the query converter, and the second query vector sequence obtained by updating the last training batch is used as the first query vector sequence of the first training batch of the molecular marker.

[0343] In some embodiments, the second training module 1332 is further configured to perform a first autoregressive training task on the pre-trained language understanding model based on the molecular marker sequence, molecular representation data, and prompts, to obtain a first language understanding model for generating the molecular description text; and to perform a second autoregressive training task on the first language understanding model based on the molecular marker sequence, the prompts, and the molecular description text, to obtain a second language understanding model for generating the molecular description text and for generating the molecular marker sequence, wherein the second language understanding model serves as the molecular data processing model.

[0344] In some embodiments, the second training module 1332 is further configured to concatenate the molecular marker sequence, the embedded representation of the molecular representation data, the embedded representation of the prompt, and the embedded representation of the molecular description text to obtain a first multimodal sequence; mask the embedded representation of the molecular description text in the first multimodal sequence to obtain a first input sequence; perform a first prediction process on the masked embedded representation of the molecular description text in the first input sequence using the pre-trained language understanding model to obtain a first prediction result; determine a first prediction loss value based on the first prediction result and the embedded representation of the molecular description text; and update the pre-trained language understanding model based on the first prediction loss value to obtain the first language understanding model for generating the molecular description text.

[0345] In some embodiments, the second training module 1332 is further configured to concatenate the molecular marker sequence, the embedded representation of the prompt, and the embedded representation of the molecular description text to obtain a second multimodal sequence; mask the molecular marker sequence in the second multimodal sequence to obtain a second input sequence; perform a second prediction process on the molecular marker sequence using the first language understanding model to obtain a second prediction result; determine a second prediction loss value based on the second prediction result and the molecular marker sequence; and update the first language understanding model based on the second prediction loss value to obtain the second language understanding model used to generate the molecular description text and the second language understanding model used to generate the molecular marker sequence.

[0346] In some embodiments, the second training module 1332 is further configured to perform a first autoregressive training task on the pre-trained language understanding model based on the molecular marker sequence, molecular representation data, and prompts to obtain a first molecular data processing model for generating the molecular description text; and to perform a second autoregressive training task on the pre-trained language understanding model based on the molecular marker sequence, the prompts, and the molecular description text to obtain a second molecular data processing model for generating the molecular marker sequence.

[0347] In some embodiments, the second training module 1332 is further configured to acquire training data for a specific task related to molecular processing, wherein the training data for the specific task includes molecular data samples and labeled data samples; perform specific task result prediction processing on the molecular data samples using the molecular data processing model to obtain predicted data; determine a specific task loss value based on the labeled data samples and the predicted data; and update the molecular data processing model based on the specific task loss value to obtain a molecular data processing model for the specific task.

[0348] The following description continues to illustrate the exemplary structure of the molecular data processing device 134 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2B As shown, the software modules stored in the molecular data processing device 134 in the memory 130-2 may include:

[0349] The data processing module 1341 is used to acquire molecular data to be processed and processing prompts.

[0350] In some embodiments, the data processing module 1341 is further configured to, in response to the molecular structure represented by the molecular data to be processed, perform molecular marker generation processing on the molecular data to be processed by a molecular marker generator to obtain the molecular marker sequence of the molecular data to be processed.

[0351] The prediction module 1342 is used to perform molecular description text prediction processing based on the molecular marker sequence of the molecular data to be processed and the embedded representation of the prompt to be processed through a molecular data processing model to obtain the molecular description text of the molecular data to be processed. The molecular marker and the molecular data processing model are trained by the training method of the molecular data processing model provided in the embodiments of this application.

[0352] The following description continues to illustrate the exemplary structure of the molecular data processing device 135 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2C As shown, the software modules stored in the molecular data processing device 135 of the memory 130-3 may include:

[0353] The data processing module 1351 is used to acquire molecular data to be processed and processing prompts.

[0354] The prediction module 1352 is used to respond to the fact that the molecular data to be processed is text data, and to perform molecular marker prediction processing based on the embedding representation of the molecular data to be processed and the embedding representation of the prompt to be processed through the molecular data processing model to obtain the molecular marker sequence of the molecular data to be processed. The molecular marker and the molecular data processing model are trained by the training method of the molecular data processing model provided in the embodiments of this application.

[0355] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. An electronic device's processor reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the molecular data processing model training method or molecular data processing method described above in this application.

[0356] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the training method or speech synthesis method of the speech synthesis model provided in this application. For example, ... Figure 3A The training method of the molecular data processing model shown or Figure 4 The molecular data processing method is shown.

[0357] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0358] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0359] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0360] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0361] In summary, through the embodiments of this application, multiple training batches are performed on the molecular marker, feature interaction processing is carried out on the molecular graph data and molecular description text, the first query vector sequence and the encoding are continuously updated, and the molecular graph data is mapped to the molecular marker sequence using the interaction query sequence obtained by feature interaction processing, so that the molecular representation sequence and the molecular description text representation are aligned, thereby improving the consistency of multimodal data representation. This unified representation eliminates the encoding differences between different modalities, enabling the trained molecular data processing model to process and fuse multimodal information more naturally, accurately capture and align the deep relationship between molecules and text, and achieve the beneficial effect of improving the generalization ability of the molecular data processing model.

[0362] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A training method for a molecular data processing model, characterized in that, The molecular data processing model is trained with the aid of molecular markers; the method includes: Multiple training batches are iteratively performed on the molecular marker, with the following processing performed in each training batch: The molecular marker is used to perform feature interaction processing on the molecular description text, molecular graph data, and the first query vector sequence of the current training batch to obtain an interactive query sequence. Based on the interactive query sequence, the current training batch's encoding is queried to obtain the molecular marker sequence. A first loss value is determined based on the interactive query sequence and the molecular marker sequence. The molecular marker, the first query vector sequence, and the codebook are updated based on the first loss value, wherein the updated molecular marker, the first query vector sequence, and the codebook are used for the next training batch of the molecular marker; Based on the molecular markers after the multiple training batches, the training task of the molecular data processing model is executed; The feature interaction processing is implemented through a pre-trained query converter in the molecular marker, and the following processing is performed in each training batch of the query converter: The query converter performs a first attention process on the text feature samples of the molecular description text sample and the second query vector sequence of the current training batch of the query converter to obtain a first attention feature sample. When the current training batch of the query converter is the first training batch, the second query vector sequence is randomly initialized. The query converter performs second attention processing on the encoded feature samples of the first attention feature sample and the molecular graph data sample to obtain the second attention feature sample. The second attention feature sample is processed by the query converter to obtain an interactive query sequence sample. Multiple training tasks are performed based on the interactive query sequence samples and the text feature samples to obtain the pre-trained query converter.

2. The method according to claim 1, characterized in that, The step involves performing feature interaction processing on the molecular description text, molecular graph data, and the first query vector sequence of the current training batch using the molecular marker to obtain an interactive query sequence, including: The text features of the molecular description text and the first query vector sequence are subjected to a first attention process to obtain the first attention features; The first attention feature and the encoded features of the molecular graph data are subjected to a second attention process to obtain a second attention feature; The second attention feature is processed by feedforward to obtain the interactive query sequence.

3. The method according to claim 2, characterized in that, The first attention processing is performed on the text features of the molecular description text and the first query vector sequence to obtain the first attention features, including: Based on the textual features of the molecular description text and the first query vector sequence, a first attention score matrix is ​​determined; Obtain the mask matrix of the first attention score matrix; The first attention score matrix is ​​masked based on the mask matrix, and the resulting mask score matrix is ​​normalized to obtain the first attention weight matrix. The first attention feature is obtained by weighting and summing the text features using the first attention weight matrix.

4. The method according to claim 1, characterized in that, The interactive query sequence includes multiple interactive query vectors. The step of querying the encoder of the current training batch based on the interactive query sequence to obtain the molecular marker sequence includes: From the encoding of the current training batch, query the nearest encoding vector of each interaction query vector, wherein the nearest encoding vector is the encoding vector with the smallest distance to the interaction query vector, and query the index of each nearest encoding vector; The molecular marker sequence is obtained by combining the indices of each of the most recently encoded vectors.

5. The method according to any one of claims 1 to 4, characterized in that, The molecular marker sequence includes multiple molecular markers; determining the first loss value based on the interactive query sequence and the molecular marker sequence includes: Query the encoding vector indexed by each molecular marker from the encoding book of the current training batch, and combine the encoding vectors corresponding to each molecular marker into a first embedding feature; The first embedded feature is subjected to feature mapping processing to obtain the mapped feature; The molecular representation data is embedded and encoded to obtain the second embedding feature; A first loss value is determined based on the mapping feature, the second embedding feature, the interactive query sequence, and the first embedding feature.

6. The method according to claim 5, characterized in that, Determining the first loss value based on the mapping feature, the second embedding feature, the interactive query sequence, and the first embedding feature includes: Based on the difference between the mapping feature and the second embedding feature, the alignment loss is determined; Based on the first difference between the interactive query sequence and the first embedded feature, the encoding loss is determined; Based on the second difference between the interactive query sequence and the first embedded feature, the consistency loss is determined; A first loss value is determined based on the alignment loss, the codebook loss, and the consistency loss.

7. The method according to claim 1, characterized in that, The multiple training tasks include a molecular text contrast learning training task, a molecular text matching training task, and a molecular text generation training task. The process of performing multiple training tasks based on the interactive query sequence samples and the text feature samples to obtain the pre-trained query converter includes: Based on the interactive query sequence samples and the text feature samples, determine the first sub-loss value corresponding to the molecular text contrast learning training task; Obtain the text embedding feature samples of the molecular description text sample; Based on the interactive query sequence sample, the encoded feature sample, and the text embedding feature sample, determine the second sub-loss value corresponding to the molecular text matching training task; Based on the text embedding feature samples and the encoded feature samples, the third sub-loss value corresponding to the molecular text generation training task is determined; Based on the first sub-loss value, the second sub-loss value, and the third sub-loss value, the query converter and the second query vector sequence of the current training batch are updated. The updated query converter and the second query vector sequence are used for the training of the next batch of the query converter. The second query vector sequence obtained from the last training batch is used as the first query vector sequence of the first training batch of the molecular marker.

8. The method according to claim 1, characterized in that, The molecular labeler, based on the multiple training batches, performs the training task of the molecular data processing model, including: Based on the molecular marker sequence, molecular representation data and prompts, the pre-trained language understanding model is subjected to a first autoregressive training task to obtain a first language understanding model for generating the molecular description text. Based on the molecular marker sequence, the prompt, and the molecular description text, the first language understanding model is subjected to a second autoregressive training task to obtain a second language understanding model for generating the molecular description text and for generating the molecular marker sequence, wherein the second language understanding model serves as the molecular data processing model.

9. The method according to claim 8, characterized in that, The step of performing a first autoregressive training task on the pre-trained language understanding model based on the molecular marker sequence, molecular representation data, and prompts to obtain a first language understanding model for generating the molecular description text includes: By concatenating the molecular marker sequence, the embedded representation of the molecular representation data, the embedded representation of the prompt, and the embedded representation of the molecular description text, a first multimodal sequence is obtained. The embedded representation of the molecular description text in the first multimodal sequence is masked to obtain the first input sequence; The pre-trained language understanding model is used to perform a first prediction process on the embedded representation of the masked molecular description text in the first input sequence to obtain a first prediction result. The first prediction loss value is determined based on the first prediction result and the embedded representation of the molecular description text; The pre-trained language understanding model is updated based on the first predicted loss value to obtain the first language understanding model used to generate the molecular description text. The step of performing a second autoregressive training task on the first language understanding model based on the molecular marker sequence, the prompt, and the molecular description text to obtain a second language understanding model for generating the molecular description text and for generating the molecular marker sequence includes: The molecular marker sequence, the embedded representation of the prompt, and the embedded representation of the molecular description text are concatenated to obtain the second multimodal sequence; The molecular marker sequence in the second multimodal sequence is masked to obtain the second input sequence; The molecular marker sequence is subjected to a second prediction process using the first language understanding model to obtain a second prediction result. A second prediction loss value is determined based on the second prediction result and the molecular marker sequence; The first language understanding model is updated based on the second predicted loss value to obtain the second language understanding model used to generate the molecular description text and the molecular marker sequence.

10. The method according to claim 1, characterized in that, The molecular labeler, based on the multiple training batches, performs the training task of the molecular data processing model, including: Based on the molecular marker sequence, molecular representation data and prompts, the pre-trained language understanding model is subjected to a first autoregressive training task to obtain a first molecular data processing model for generating the molecular description text. Based on the molecular marker sequence, the prompt, and the molecular description text, a second autoregressive training task is performed on the pre-trained language understanding model to obtain a second molecular data processing model for generating the molecular marker sequence.

11. The method according to any one of claims 8 to 10, characterized in that, After the molecular labeler based on the multiple training batches performs the training task of the molecular data processing model, the method further includes: Acquire training data for a specific task related to molecular processing, wherein the training data for the specific task includes molecular data samples and labeled data samples; The molecular data processing model is used to perform specific task result prediction processing on the molecular data samples to obtain prediction data. Based on the labeled data samples and the predicted data, determine the loss value for a specific task; The molecular data processing model is updated based on the loss value of the specific task to obtain a molecular data processing model for the specific task.

12. A molecular data processing method, characterized in that, The method includes: Obtain the molecular data to be processed and the processing prompts; In response to the molecular structure represented by the molecular data to be processed, molecular marker generation processing is performed on the molecular data to be processed by a molecular marker generator to obtain the molecular marker sequence of the molecular data to be processed. Molecular descriptive text prediction processing is performed on the molecular label sequence of the molecular data to be processed and the embedded representation of the prompt to be processed by the molecular data processing model to obtain the molecular descriptive text of the molecular data to be processed, wherein the molecular labeler and the molecular data processing model are trained by the method of any one of claims 1 to 10.

13. A molecular data processing method, characterized in that, The method includes: Obtain the molecular data to be processed and the processing prompts; In response to the fact that the molecular data to be processed is text data, molecular marker prediction processing is performed based on the embedded representation of the molecular data to be processed and the embedded representation of the prompt to be processed through a molecular data processing model to obtain the molecular marker sequence of the molecular data to be processed, wherein the molecular data processing model is trained by the method of any one of claims 1 to 10.

14. A training device for a molecular data processing model, characterized in that, The device includes: The first training module is used to iteratively execute multiple training batches on the molecular marker, wherein the following processes are performed in each training batch: the molecular marker performs feature interaction processing on the molecular description text, molecular graph data and the first query vector sequence of the current training batch to obtain an interaction query sequence; the codebook of the current training batch is queried based on the interaction query sequence to obtain a molecular marker sequence; a first loss value is determined based on the interaction query sequence and the molecular marker sequence; the molecular marker, the first query vector sequence and the codebook are updated based on the first loss value, wherein the updated molecular marker, the first query vector sequence and the codebook are used for the next training batch of the molecular marker; The second training module is used to perform the training task of the molecular data processing model based on the molecular markers after the multiple training batches. The feature interaction processing is implemented through a pre-trained query converter in the molecular marker. The first training module is further configured to perform the following processing in each training batch of the query converter: The query converter performs a first attention process on the text feature samples of the molecular description text sample and the second query vector sequence of the current training batch of the query converter to obtain a first attention feature sample. When the current training batch of the query converter is the first training batch, the second query vector sequence is randomly initialized. The query converter performs second attention processing on the encoded feature samples of the first attention feature sample and the molecular graph data sample to obtain the second attention feature sample. The second attention feature sample is processed by the query converter to obtain an interactive query sequence sample. Multiple training tasks are performed based on the interactive query sequence samples and the text feature samples to obtain the pre-trained query converter.

15. The training apparatus for the molecular data processing model according to claim 14, characterized in that, The first training module is also used for: The text features of the molecular description text and the first query vector sequence are subjected to a first attention process to obtain the first attention features; The first attention feature and the encoded features of the molecular graph data are subjected to a second attention process to obtain a second attention feature; The second attention feature is processed by feedforward to obtain the interactive query sequence.

16. The training apparatus for the molecular data processing model according to claim 15, characterized in that, The first training module is also used for: Based on the textual features of the molecular description text and the first query vector sequence, a first attention score matrix is ​​determined; Obtain the mask matrix of the first attention score matrix; The first attention score matrix is ​​masked based on the mask matrix, and the resulting mask score matrix is ​​normalized to obtain the first attention weight matrix. The first attention feature is obtained by weighting and summing the text features using the first attention weight matrix.

17. The training apparatus for the molecular data processing model according to claim 14, characterized in that, The interactive query sequence includes multiple interactive query vectors, and the first training module is further configured to: From the encoding of the current training batch, query the nearest encoding vector of each interaction query vector, wherein the nearest encoding vector is the encoding vector with the smallest distance to the interaction query vector, and query the index of each nearest encoding vector; The molecular marker sequence is obtained by combining the indices of each of the most recently encoded vectors.

18. The training apparatus for a molecular data processing model according to any one of claims 14 to 17, characterized in that, The molecular marker sequence includes multiple molecular markers; the first training module is further used for: Query the encoding vector indexed by each molecular marker from the encoding book of the current training batch, and combine the encoding vectors corresponding to each molecular marker into a first embedding feature; The first embedded feature is subjected to feature mapping processing to obtain the mapped feature; The molecular representation data is embedded and encoded to obtain the second embedding feature; A first loss value is determined based on the mapping feature, the second embedding feature, the interactive query sequence, and the first embedding feature.

19. The training apparatus for the molecular data processing model according to claim 18, characterized in that, The first training module is also used for: Based on the difference between the mapping feature and the second embedding feature, the alignment loss is determined; Based on the first difference between the interactive query sequence and the first embedded feature, the encoding loss is determined; Based on the second difference between the interactive query sequence and the first embedded feature, the consistency loss is determined; A first loss value is determined based on the alignment loss, the codebook loss, and the consistency loss.

20. The training apparatus for the molecular data processing model according to claim 14, characterized in that, The multiple training tasks include molecular text comparison learning training tasks, molecular text matching training tasks, and molecular text generation training tasks. The first training module is further used for: Based on the interactive query sequence samples and the text feature samples, determine the first sub-loss value corresponding to the molecular text contrast learning training task; Obtain the text embedding feature samples of the molecular description text sample; Based on the interactive query sequence sample, the encoded feature sample, and the text embedding feature sample, determine the second sub-loss value corresponding to the molecular text matching training task; Based on the text embedding feature samples and the encoded feature samples, the third sub-loss value corresponding to the molecular text generation training task is determined; Based on the first sub-loss value, the second sub-loss value, and the third sub-loss value, the query converter and the second query vector sequence of the current training batch are updated. The updated query converter and the second query vector sequence are used for the training of the next batch of the query converter. The second query vector sequence obtained from the last training batch is used as the first query vector sequence of the first training batch of the molecular marker.

21. The training apparatus for the molecular data processing model according to claim 14, characterized in that, The second training module is also used for: Based on the molecular marker sequence, molecular representation data and prompts, the pre-trained language understanding model is subjected to a first autoregressive training task to obtain a first language understanding model for generating the molecular description text. Based on the molecular marker sequence, the prompt, and the molecular description text, the first language understanding model is subjected to a second autoregressive training task to obtain a second language understanding model for generating the molecular description text and for generating the molecular marker sequence, wherein the second language understanding model serves as the molecular data processing model.

22. The training apparatus for the molecular data processing model according to claim 21, characterized in that, The second training module is also used for: By concatenating the molecular marker sequence, the embedded representation of the molecular representation data, the embedded representation of the prompt, and the embedded representation of the molecular description text, a first multimodal sequence is obtained. The embedded representation of the molecular description text in the first multimodal sequence is masked to obtain the first input sequence; The pre-trained language understanding model is used to perform a first prediction process on the embedded representation of the masked molecular description text in the first input sequence to obtain a first prediction result. The first prediction loss value is determined based on the first prediction result and the embedded representation of the molecular description text; The pre-trained language understanding model is updated based on the first predicted loss value to obtain the first language understanding model used to generate the molecular description text. The molecular marker sequence, the embedded representation of the prompt, and the embedded representation of the molecular description text are concatenated to obtain the second multimodal sequence; The molecular marker sequence in the second multimodal sequence is masked to obtain the second input sequence; The molecular marker sequence is subjected to a second prediction process using the first language understanding model to obtain a second prediction result. A second prediction loss value is determined based on the second prediction result and the molecular marker sequence; The first language understanding model is updated based on the second predicted loss value to obtain the second language understanding model used to generate the molecular description text and the molecular marker sequence.

23. The training apparatus for the molecular data processing model according to claim 14, characterized in that, The second training module is also used for: Based on the molecular marker sequence, molecular representation data and prompts, the pre-trained language understanding model is subjected to a first autoregressive training task to obtain a first molecular data processing model for generating the molecular description text. Based on the molecular marker sequence, the prompt, and the molecular description text, a second autoregressive training task is performed on the pre-trained language understanding model to obtain a second molecular data processing model for generating the molecular marker sequence.

24. The training apparatus for a molecular data processing model according to any one of claims 21 to 23, characterized in that, The second training module is also used for: Acquire training data for a specific task related to molecular processing, wherein the training data for the specific task includes molecular data samples and labeled data samples; The molecular data processing model is used to perform specific task result prediction processing on the molecular data samples to obtain prediction data. Based on the labeled data samples and the predicted data, determine the loss value for a specific task; The molecular data processing model is updated based on the loss value of the specific task to obtain a molecular data processing model for the specific task.

25. A molecular data processing device, characterized in that, The device includes: The data processing module is used to acquire molecular data to be processed and processing prompts; The data processing module is further configured to, in response to the molecular structure represented by the molecular data to be processed, perform molecular marker generation processing on the molecular data to be processed through a molecular marker generator to obtain the molecular marker sequence of the molecular data to be processed; A prediction module is used to perform molecular description text prediction processing on the molecular data to be processed based on the molecular marker sequence of the molecular data to be processed and the embedded representation of the prompt to be processed, through a molecular data processing model, to obtain the molecular description text of the molecular data to be processed, wherein the molecular marker and the molecular data processing model are trained by the method of any one of claims 1 to 10.

26. A molecular data processing device, characterized in that, The device includes: The data processing module is used to acquire molecular data to be processed and processing prompts; A prediction module is configured to, in response to the fact that the molecular data to be processed is text data, perform molecular marker prediction processing based on the embedded representation of the molecular data to be processed and the embedded representation of the prompt to be processed through a molecular data processing model to obtain a molecular marker sequence of the molecular data to be processed, wherein the molecular data processing model is trained by the method of any one of claims 1 to 10.

27. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the training method of the molecular data processing model according to any one of claims 1 to 11, or implements the molecular data processing method according to claim 12 or claim 13.

28. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the training method of the molecular data processing model according to any one of claims 1 to 11, or the molecular data processing method according to claim 12 or claim 13.

29. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by the processor, they implement the training method of the molecular data processing model according to any one of claims 1 to 11, or the molecular data processing method according to claim 12 or claim 13.

Images