Training method and device for representation learning model, equipment and storage medium

Abstract

The invention discloses a training method and device for a representation learning model, equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring a spatial structure diagram of a sample molecule; generating a first implicit representation of the sample molecule through a neural network according to the spatial structure diagram and the feature representation of each component; obtaining a second implicit representation through the super network according to the first implicit representation; determining an attribute prediction result of atoms contained in the sample molecules through an implicit representation network according to the first implicit representation and the second implicit representation; and adjusting parameters of the representation learning model to obtain a trained representation learning model. According to the method, the second implicit expression learned by the super-network can reflect the space structure of the molecule more accurately, so that the accuracy of a molecular property prediction result in a downstream task is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, and storage medium for training a representation learning model. Background Technology

[0002] Predicting molecular properties plays an increasingly important role in the biological field, such as predicting protein function, which is crucial for drug design and disease treatment. Molecular representation learning methods have become one of the key technologies for predicting molecular properties.

[0003] Taking protein representation learning methods as an example, related technologies are mainly divided into sequence-based protein representation learning methods and structure-based protein representation learning methods. Sequence-based protein representation learning methods refer to learning the feature representation of a protein based on its amino acid sequence. Structure-based protein representation learning methods refer to learning its feature representation using the three-dimensional structural information of a protein. For example, the three-dimensional structural information of a protein is converted into 3D (three-dimensional) mesh data, and then a 3D CNN (Convolutional Neural Network) is used to learn the protein's feature representation.

[0004] The aforementioned methods, for sequence-based protein representation learning methods, suffer from inaccurate protein feature representations due to insufficient sequence information to fully reflect the complex spatial structure and functional characteristics of proteins, leading to inaccurate subsequent protein property predictions. For structure-based protein representation learning methods, the need to use 3D convolutional networks to process 3D mesh data results in high computational costs. Summary of the Invention

[0005] This application provides a training method, apparatus, device, and storage medium for representing a learning model. The technical solutions provided by this application are as follows:

[0006] According to one aspect of the embodiments of this application, a method for training a representation learning model is provided, the representation learning model including a graph neural network, an implicit representation network, and a hypernetwork, the method comprising:

[0007] Obtain a spatial structure diagram of the sample molecule, which is used to characterize the components of the sample molecule and the spatial relationships between the components;

[0008] The graph neural network generates a first implicit representation of the sample molecule based on the spatial structure diagram and the feature representations of each component. The first implicit representation is used to characterize the attribute features of the sample molecule.

[0009] The supernetwork obtains the second implicit representation based on the first implicit representation, and the second implicit representation is used to provide the downstream task network to predict the properties of the molecule;

[0010] The implicit representation network determines the property prediction results of the atoms contained in the sample molecule based on the first implicit representation and the second implicit representation.

[0011] With the goal of minimizing the error in the property prediction results of the atoms, the parameters of the representation learning model are adjusted to obtain the trained representation learning model.

[0012] According to one aspect of the embodiments of this application, a method for predicting molecular properties based on a representation learning model is provided, wherein the representation learning model includes a graph neural network, an implicit representation network, and a supernetwork, and the method includes:

[0013] Obtain a spatial structure diagram of the first molecule, which is used to characterize the components of the first molecule and the spatial relationships between the components.

[0014] The graph neural network generates a first implicit representation of the first molecule based on the spatial structure diagram and the feature representations of each component. The first implicit representation is used to characterize the attribute features of the first molecule.

[0015] The supernetwork obtains a second implicit representation based on the first implicit representation. The second implicit representation is provided to the downstream task network to predict the properties of the first molecule. The parameters of the supernetwork are obtained by training the implicit representation network.

[0016] According to one aspect of the embodiments of this application, a training apparatus for a representation learning model is provided, the representation learning model including a graph neural network, an implicit representation network, and a hypernetwork, the apparatus comprising:

[0017] An acquisition module is used to acquire a spatial structure diagram of a sample molecule, wherein the spatial structure diagram is used to characterize the components of the sample molecule and the spatial relationships between the components;

[0018] The generation module is used to generate a first implicit representation of the sample molecule based on the spatial structure diagram and the feature representations of each component through the graph neural network. The first implicit representation is used to characterize the attribute features of the sample molecule.

[0019] The module is used to obtain a second implicit representation based on the first implicit representation through the supernetwork. The second implicit representation is used to provide the downstream task network to predict the properties of the molecule.

[0020] The determination module is used to determine the property prediction results of the atoms contained in the sample molecule based on the first implicit representation and the second implicit representation through the implicit representation network;

[0021] The adjustment module is used to adjust the parameters of the representation learning model with the goal of minimizing the error of the property prediction results of the atom, so as to obtain the trained representation learning model.

[0022] According to one aspect of the embodiments of this application, a molecular property prediction device based on a representation learning model is provided, wherein the representation learning model includes a graph neural network, an implicit representation network, and a supernetwork, and the device includes:

[0023] An acquisition module is used to acquire a spatial structure diagram of a first molecule, wherein the spatial structure diagram is used to characterize the components of the first molecule and the spatial relationships between the components.

[0024] The generation module is used to generate a first implicit representation of the first molecule based on the spatial structure diagram and the feature representations of each component through the graph neural network. The first implicit representation is used to characterize the attribute features of the first molecule.

[0025] The module is configured to obtain a second implicit representation from the first implicit representation through the supernetwork. The second implicit representation is provided to the downstream task network to predict the properties of the first molecule. The parameters of the supernetwork are obtained by training the implicit representation network.

[0026] According to one aspect of the embodiments of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-described training method for a representation learning model, or the above-described method for predicting molecular properties based on a representation learning model.

[0027] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored in the storage medium, the computer program being loaded and executed by a processor to implement the above-described training method for a representation learning model, or the above-described method for predicting molecular properties based on a representation learning model.

[0028] According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including a computer program stored in a computer-readable storage medium, a processor reading from the computer-readable storage medium and executing the computer program to implement the above-described training method for a representation learning model, or the above-described method for predicting molecular properties based on a representation learning model.

[0029] The technical solutions provided in this application have at least the following beneficial effects:

[0030] On the one hand, representation learning models learn molecular feature representations based on the interactions between the components (such as atoms) and the individual feature representations of each component. This approach avoids the computational overhead associated with using 3D mesh data of molecules to learn their feature representations. On the other hand, the learning of the second implicit representation is performed through atomic-level tasks. This allows the second implicit representation learned by the supernetwork to more accurately reflect the spatial structure of the molecule, thereby improving the accuracy of molecular property predictions in downstream tasks. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the implementation environment of a solution provided in one embodiment of this application;

[0032] Figure 2 This is a flowchart of a training method for a representation learning model provided in one embodiment of this application;

[0033] Figure 3 This is a schematic diagram of the flowcharts for the training and inference phases of a representation learning model provided in one embodiment of this application;

[0034] Figure 4 This is a schematic diagram of the representation learning model network structure provided in one embodiment of this application;

[0035] Figure 5 This is a flowchart of a molecular property prediction method based on a representation learning model provided in one embodiment of this application;

[0036] Figure 6 This is a flowchart of a training method for a protein molecule representation learning model provided in one embodiment of this application;

[0037] Figure 7 This is a flowchart of a protein molecule property prediction method based on a protein molecule representation learning model provided in one embodiment of this application;

[0038] Figure 8 This is a block diagram of a training apparatus for a representation learning model provided in one embodiment of this application;

[0039] Figure 9 This is a block diagram of a molecular property prediction device based on a representation learning model provided in one embodiment of this application;

[0040] Figure 10 This is a structural block diagram of a computer device provided in one embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0042] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0043] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained model technology, operating / interactive systems, and mechatronics. Pre-trained models, also known as large-scale models or foundational models, can be widely applied to downstream tasks across various AI fields after fine-tuning. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0044] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and pre-trained learning. Pre-trained models are the latest development in deep learning, integrating all of these techniques.

[0045] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, digital twins, virtual humans, robots, AIGC (Artificial Intelligence Generated Content), conversational interaction, smart healthcare, smart customer service, and game AI. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0046] The solutions provided in this application relate to machine learning and deep learning technologies in artificial intelligence, and are specifically illustrated through the following embodiments.

[0047] Please refer to Figure 1 The diagram illustrates an implementation environment for a solution provided in one embodiment of this application. This implementation environment may include a model training device 110 and a model usage device 120.

[0048] The model training device 110 can be an electronic device such as a mobile phone, desktop computer, tablet computer, laptop computer, vehicle terminal, server, intelligent robot, smart TV, multimedia playback device, or other electronic devices with strong computing power; this application does not limit this. The model training device 110 is used to train the representation learning model.

[0049] In this embodiment, the representation learning model is a deep neural network model. Optionally, the model training device 110 can train the representation learning model using machine learning to achieve better performance. Optionally, the training process of the representation learning model is as follows (this is only a brief description; the specific training process is described in the following embodiments): obtaining the spatial structure map of the sample molecule; generating a first implicit representation of the sample molecule using a graph neural network based on the spatial structure map and the feature representations of each component; obtaining a second implicit representation using a supernetwork based on the first implicit representation, the second implicit representation being provided to the downstream task network to predict the properties of the molecule; determining the property prediction results of the atoms contained in the sample molecule using the implicit representation network based on the first and second implicit representations; adjusting the parameters of the representation learning model with the goal of minimizing the error of the atom property prediction results to obtain the trained representation learning model.

[0050] The device 120 for using the model can be an electronic device such as a mobile phone, desktop computer, tablet computer, laptop computer, vehicle terminal, server, intelligent robot, smart TV, multimedia playback device, or other electronic devices with strong computing power; this application does not limit this. The device 120 for using the model can employ a trained representation learning model to predict the properties of molecules.

[0051] The model training device 110 and the model usage device 120 can be two separate devices or the same device.

[0052] The method provided in this application embodiment can be executed by a computer device, which refers to an electronic device with data computing, processing, and storage capabilities. When the computer device is a server, the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The computer device can be... Figure 1 The model training device 110 can also be the model usage device 120.

[0053] Please refer to Figure 2 The diagram illustrates a flowchart of a training method for a representation learning model according to an embodiment of this application. The execution entity for each step of the method can be a computer device; for example, the computer device can be... Figure 1 The illustrated scheme is implemented in a model training device 110 within an environment. The method may include at least one of the following steps 210 to 250.

[0054] Step 210: Obtain the spatial structure diagram of the sample molecule. The spatial structure diagram is used to characterize the components of the sample molecule and the spatial relationships between the components.

[0055] A molecule is a stable compound formed by atoms bonded together by chemical bonds. For example, molecules can be proteins, lipids, sugars, nucleic acids, etc.

[0056] A molecular spatial structure diagram refers to the arrangement and organization of molecules in three-dimensional space. It shows the relative positions of the constituent parts of a molecule, the types and lengths of chemical bonds, and the overall three-dimensional configuration of the molecule. Spatial structure diagrams enable a better understanding of the properties and functions of molecules, as well as their mechanisms of action in biological systems or chemical reactions.

[0057] The components of a molecule refer to the chemical elements or functional groups that make up the molecule. Chemical elements refer to the types of atoms that make up a molecule; functional groups, also known as atomic groups, refer to the collection of multiple atoms in a molecule that have specific chemical properties.

[0058] In some embodiments, an atom set can refer to the basic building blocks of a molecule. For example, when the molecule is a protein, the atom set can be an amino acid (or amino acid residue); when the molecule is a sugar, the atom set can be a monosaccharide; and when the molecule is a nucleic acid, the atom set can be a nucleotide. In some embodiments, an atom set can also refer to other groups composed of multiple atoms that have specific chemical functions, such as hydroxyl or amino groups, and this application does not limit this.

[0059] In some embodiments, the components of a molecule may include atoms and groups of atoms. Therefore, the spatial structure of a molecule can be described at both the atomic and group levels of granularity, i.e., the spatial structure diagram includes atomic diagrams and group diagrams; each vertex in the atomic diagram represents an atom in the sample molecule, and an edge connecting two vertices indicates an interaction between the atoms corresponding to the two vertices; each vertex in the group diagram represents a group of atoms in the sample molecule, and an edge connecting two vertices indicates an interaction between the groups of atoms corresponding to the two vertices.

[0060] The interaction between atoms or groups of atoms here refers to the chemical bonds between them, such as covalent bonds, hydrogen bonds, and ionic bonds. Atoms or groups of atoms are connected by chemical bonds to form specific spatial structures.

[0061] In some embodiments, molecules can be separated into multiple atomic groups using chemical analysis methods. These methods include, but are not limited to, mass spectrometry, nuclear magnetic resonance (NMR) spectroscopy, and chromatography. These techniques can separate and identify molecules based on the physical and chemical properties of the atomic groups within them, thereby obtaining the corresponding atomic groups for that molecule.

[0062] For example, when the molecule is a protein, the spatial structure diagram of the protein may include an atomic diagram and an amino acid diagram; each vertex in the atomic diagram represents an atom in the protein, and an edge connecting two vertices indicates that there is an interaction between the atoms corresponding to the two vertices; each vertex in the amino acid diagram represents an amino acid in the protein, and an edge connecting two vertices indicates that there is an interaction between the amino acids corresponding to the two vertices, wherein an amino acid is composed of at least two atoms.

[0063] The above method describes the spatial structure of molecules at two different granularities. The atomic diagram illustrates the positions and interactions of atoms within the molecule through chemical bonds, while the atomic group diagram illustrates the spatial arrangement and interactions of atomic groups through chemical bonds. This approach facilitates subsequent graph neural networks in learning molecular structural information at different levels.

[0064] Step 220: Based on the spatial structure diagram and the feature representations of each component, a first implicit representation of the sample molecule is generated using a graph neural network. The first implicit representation is used to characterize the attribute features of the sample molecule.

[0065] The spatial structure diagram described above is used to reflect the interaction relationships between atoms or groups of atoms. In some embodiments, adjacency matrices, adjacency lists, or other graph structure representation methods can be used to characterize the spatial structure diagram. For example, an adjacency matrix is ​​used to characterize the spatial structure diagram, where each element a in the adjacency matrix... i,j i and j are positive integers, representing whether there is an interaction relationship between the i-th atom or group of atoms and the j-th atom or group of atoms. They are usually binary values, such as using a binary value of 1 to indicate that there is an interaction relationship between the two and a binary value of 0 to indicate that there is no interaction relationship between the two; or using a binary value of 0 to indicate that there is an interaction relationship between the two and a binary value of 1 to indicate that there is no interaction relationship between the two. This application does not limit this.

[0066] In some embodiments, the characteristic representations of atoms and atomic groups can be determined based on the properties of atoms and atomic groups. These properties may include the type of atom or atomic group, chemical bond information, charge, electronegativity, atomic weight, covalent radius, etc., which are not limited herein.

[0067] In some embodiments, the feature representations of atoms and atomic groups can be determined based on the properties of atoms and atomic groups using quantum chemical methods (such as density functional theory), molecular force field methods (such as molecular force field methods, force field optimization methods), machine learning models (such as neural networks, support vector machines), or statistical methods (such as principal component analysis, cluster analysis). This application does not limit the methods used to obtain the respective feature representations of atoms and atomic groups.

[0068] Graph Neural Networks (GNNs) are neural network models used to process graph-structured data. They can effectively learn and represent the relationships between nodes and edges in graph-structured data, and are suitable for processing and analyzing various types of graph-structured data.

[0069] In some embodiments, for the atomic graph described above, the GNN can learn the interaction relationships between atoms and encode the learned information as an implicit representation; for the atomic group graph described above, the GNN can learn the interaction relationships between atomic groups and encode the learned information as an implicit representation.

[0070] In some embodiments, the first implicit representation includes a global implicit representation and a local implicit representation. The global implicit representation is determined based on the feature representations of multiple atoms of the sample molecule, and the local implicit representation is determined based on the feature representations of multiple groups of atoms of the sample molecule, wherein each group of atoms includes at least two atoms.

[0071] In some embodiments, the global implicit representation is determined based on the atomic graph, that is, the global implicit representation is generated by a graph neural network based on the atomic graph and the feature representation of each atom contained in the sample molecule.

[0072] In some embodiments, for a plurality of atoms contained in a molecule, local implicit representations of each atom are determined; a global implicit representation is generated based on the local implicit representations of the atom. The local implicit representation of an atom is determined based on that atom and other atoms connected to it.

[0073] In some embodiments, determining the local implicit representation of each of the multiple atoms contained in a molecule may include: for each of the multiple atoms, determining the local implicit representation of the atom using a graph neural network based on the feature representation of the atom and the feature representations of the other atoms connected to the atom.

[0074] In some embodiments, generating a global implicit representation based on the local implicit representations of the multiple atoms may include: using aggregation functions, such as pooling operations, attention mechanisms, etc., to generate a global implicit representation based on the local implicit representations of the multiple atoms.

[0075] The aforementioned method, because the global implicit representation is determined based on the characteristic representations of multiple atoms, can capture the structural characteristics of the entire molecule from the perspective of atomic-level structural information. This method, starting from the atomic level, allows the learned implicit representation to more accurately reflect the microscopic and macroscopic structural features of the molecule, including its spatial configuration, the type and length of chemical bonds, and the relative positions of atoms, providing crucial information for a deeper understanding of the physicochemical properties of molecules.

[0076] In some embodiments, the local implicit representation is determined based on the atomic group graph, that is, the local implicit representation is generated by a graph neural network according to the atomic group graph and the feature representation of each atomic group contained in the sample molecule.

[0077] In some embodiments, for any one of the atomic groups contained in the sample molecule, a local implicit vector of that atomic group is generated based on that atomic group and the other atomic groups connected to it.

[0078] In some embodiments, generating a local implicit vector for an atom group based on the atom group and other atom groups connected to it may include generating a local implicit representation of the atom group based on the feature representation of the atom group and the feature representations of the other atom groups connected to it. Each atom group corresponds to a local implicit representation.

[0079] The aforementioned method uses local implicit representations determined based on the characteristic representations of multiple atomic groups. These atomic groups often possess specific spatial configurations, atomic types, and reflect specific physical or chemical properties of the molecule. By focusing on the microstructure and local environment of atomic groups, this method effectively reveals local interactions and functional property changes within the molecule, facilitating subsequent predictions of atomic properties.

[0080] Step 230: The supernetwork obtains the second implicit representation based on the first implicit representation. The second implicit representation is used to provide the downstream task network to predict the properties of the molecule.

[0081] In some embodiments, the hypernetwork hw(·) is constructed based on an MLP (Multilayer Perceptron). Specifically, an MLP is a feedforward neural network, typically composed of multiple fully connected layers (i.e., each neuron is connected to all neurons in the previous layer). Its construction process includes: an input layer, which receives the first implicit representation of the molecule; hidden layers, comprising multiple fully connected layers, each consisting of multiple neurons. Each neuron computes the output through a linear combination of activation functions, commonly including ReLU (Rectified Linear Unit); and an output layer, where the output is a second implicit representation. The neurons in the output layer convert the first implicit representation into the second implicit representation using appropriate activation functions, such as softmax, sigmoid, or linear activation functions.

[0082] In some embodiments, obtaining the second implicit representation through a hypernetwork based on the first implicit representation includes: through the hypernetwork h w (·) Based on the global implicit representation u global The second implicit representation ζ is obtained by referring to Formula 1 below:

[0083] ζ=h w (u global ) (Formula 1)

[0084] Where w refers to the weight parameters of the hypernetwork, and h is the hypernetwork weight parameter. w(·) Maps the global implicit representation to a second implicit representation ζ. During the training phase, by optimizing the supernetwork weight parameters w, a better second implicit representation can be obtained. This better second implicit representation is used as input to the downstream task network to predict the properties of molecules.

[0085] Downstream task networks refer to neural networks used to predict molecular properties. These networks receive a second implicit representation as input and, through learning and analysis, delve deeper into the molecular structural information and functional characteristics within this representation, thus outputting predicted molecular properties such as function, activity, solubility, or biological activity. Because the parameters of the GNN and the weight parameters w of the supernetwork are optimized, the second implicit representation ζ input to the downstream network can more effectively capture the molecular structural information and functional characteristics, thereby improving the accuracy of the downstream task network's predictions.

[0086] In some embodiments, this application optimizes the GNN parameters and hypernetwork parameters w by designing a self-supervised task. This self-supervised task is an atomic property prediction task, specifically comprising steps 240 and 250 below.

[0087] Step 240: Using an implicit representation network, determine the property prediction results of the atoms contained in the sample molecule based on the first implicit representation and the second implicit representation.

[0088] like Figure 3 As shown, during the training phase of the representation learning model, the GNN obtains a first implicit representation based on the sample molecules; the supernetwork determines a second implicit representation based on the first implicit representation; the implicit representation network determines the attribute prediction results of atoms based on the first and second implicit representations; based on these attribute prediction results, the parameters of the GNN and the supernetwork are adjusted and optimized through backpropagation to obtain the adjusted representation learning model. Since the adjusted GNN and supernetwork parameters are optimal, the second implicit representation obtained based on the adjusted GNN and supernetwork is also optimal, thus improving the accuracy of downstream task predictions.

[0089] The properties of an atom are used to describe its chemical characteristics or structural information, and may include the type of atom, chemical bond information, charge, electronegativity, atomic weight, covalent radius, etc. This application does not limit these properties.

[0090] In some embodiments, determining the property prediction results of atoms contained in a sample molecule by using an implicit representation network based on a first implicit representation and a second implicit representation includes: determining the property prediction results of atoms by using an implicit representation network based on a local implicit representation, a second implicit representation, and the spatial coordinates of atoms contained in the sample molecule.

[0091] The spatial coordinates of an atom are the mathematical representation of the atom's position in three-dimensional space, which can be denoted as (x, y, z), where x, y, and z are real numbers.

[0092] The aforementioned implicit representation network can also be called an implicit neural representation (INR) network. For any atom in a sample molecule, in the INR network, the features of each atom (such as atom type) are encoded into a continuous function. This continuous function can determine the attribute prediction result of the atom corresponding to that spatial coordinate, such as predicting the atom type, based on the atom's spatial coordinates.

[0093] The aforementioned continuous function can be constructed based on a Convolutional Neural Network (CNN) model. Specifically, this CNN model can include convolutional layers, pooling layers, and fully connected layers. Convolutional layers extract the structural and functional features of molecules from the input data using convolutional operations; pooling layers reduce the spatial size of the convolutional layer output while preserving important feature information; finally, fully connected layers perform classification or regression prediction. For example, in this application, fully connected layers can output the predicted atom type classification result.

[0094] In some embodiments, such as Figure 4 As shown, the INR network can include L convolutional layers, such as... Figure 4 Convolutional layers 41, 42, and 43, wherein the second implicit representation ζ comprises L implicit representations, which can be represented as ζ = {ζ1, ζ2, ..., ζ} l}, where L is a positive integer. Here, each of the L implicit representations serves as the input to one of the L convolutional layers. It can be understood that these L implicit representations correspond one-to-one with the L convolutional layers in the INR network. For example, the first implicit representation serves as the input to the first convolutional layer in the INR network.

[0095] Step 250: With the goal of minimizing the error of the atom property prediction results, adjust the parameters of the representation learning model to obtain the trained representation learning model.

[0096] The error in the predicted atom properties refers to the error between the predicted atom properties and the actual atom properties. For example, the atom property can be the atom type; that is, the parameters of the representation learning model are adjusted with the goal of minimizing the error between the predicted atom type and the actual atom type.

[0097] In some embodiments, this application also designs another self-supervised task to optimize the hypernetwork parameters. This self-supervised task is a spatial coordinate denoising task, which generates spatial coordinate prediction results of atoms through an implicit representation network based on a first implicit representation, a second implicit representation, and the spatial coordinates of atoms contained in the sample molecule.

[0098] In some embodiments, generating spatial coordinate prediction results of atoms through an implicit representation network based on a first implicit representation, a second implicit representation, and the spatial coordinates of atoms contained in the sample molecule may include: generating spatial coordinate prediction results of atoms through an implicit representation network based on a local implicit representation, a second implicit representation, and the spatial coordinates of atoms contained in the sample molecule.

[0099] In some embodiments, the parameters of the representation learning model are adjusted with the goal of minimizing the error of the spatial coordinate prediction results of atoms, resulting in a trained representation learning model.

[0100] The two self-supervised tasks described above, on the one hand, enable the learning of molecular spatial structure information at the atomic level, thereby enhancing the accuracy of the second implicit representation in expressing molecular spatial structure and further improving the prediction accuracy of downstream task networks. On the other hand, through the spatial coordinate denoising task, the model can learn stable molecular structures from noisy data, enhancing its robustness to noise and its ability to handle sparse data. Furthermore, these two self-supervised tasks can learn molecular spatial structure at the atomic level, and the second implicit representation is also provided to downstream task networks, improving the model's generalization ability in downstream task networks.

[0101] The technical solution provided in this application, on the one hand, uses a representation learning model to learn the feature representation of a molecule based on the interaction relationships between its components (such as atoms) and the feature representations of each component. This method avoids the problem of high computational overhead caused by using 3D mesh data of the molecule to learn its feature representation. On the other hand, the learning of the second implicit representation is trained through atomic-level tasks, which enables the second implicit representation learned by the supernetwork to more accurately reflect the spatial structure of the molecule, thereby improving the accuracy of molecular property prediction results in downstream tasks.

[0102] Please refer to Figure 5 This document illustrates a flowchart of a molecular property prediction method based on a representation learning model, provided in one embodiment of this application. The execution entity for each step of this method can be a computer device; for example, the computer device could be... Figure 1 The model in the illustrated implementation environment uses device 120. The method may include at least one of the following steps 510 to 530.

[0103] Step 510: Obtain the spatial structure diagram of the first molecule. The spatial structure diagram is used to characterize the components of the first molecule and the spatial relationships between the components.

[0104] In some embodiments, the first implicit representation includes a global implicit representation and a local implicit representation. The global implicit representation is determined based on the characteristic representations of each of the plurality of atoms of the first molecule, and the local implicit representation is determined based on the characteristic representations of each of the plurality of groups of atoms of the first molecule, wherein the groups of atoms include at least two atoms.

[0105] In some embodiments, the spatial structure diagram includes an atomic diagram and an atomic group diagram; each vertex in the atomic diagram represents an atom in the first molecule, and an edge connecting two vertices indicates that there is an interaction relationship between the atoms corresponding to the two vertices; each vertex in the atomic group diagram represents a group of atoms in the first molecule, and an edge connecting two vertices indicates that there is an interaction relationship between the groups of atoms corresponding to the two vertices.

[0106] In some embodiments, a global implicit representation is generated by a graph neural network based on the atomic graph and the feature representations of each atom contained in the first molecule; a local implicit representation is generated by a graph neural network based on the atomic group graph and the feature representations of each atomic group contained in the first molecule.

[0107] The "first molecule" mentioned above refers to any single molecule. For a more detailed explanation, please refer to the relevant description in step 210 above.

[0108] Step 520: Generate a first implicit representation of the first molecule using a graph neural network based on the spatial structure diagram and the feature representations of each component. The first implicit representation is used to characterize the attribute features of the first molecule.

[0109] For a more detailed explanation, please refer to the relevant description of step 220 above.

[0110] Step 530: The super network obtains the second implicit representation based on the first implicit representation. The second implicit representation is used to provide the downstream task network to predict the properties of the first molecule. The parameters of the super network are obtained by training the implicit representation network.

[0111] In some embodiments, the parameters of the supernetwork are obtained by training an implicit representation network. The training process may include: determining the predicted properties of atoms contained in a sample molecule using the implicit representation network based on a first implicit representation and a second implicit representation; adjusting the parameters of the representation learning model with the objective of minimizing the error in the predicted atomic properties, resulting in a trained representation learning model, including a trained GNN network and a trained supernetwork. For specific training methods, please refer to the training process described above. Figure 3As shown, during the training phase of the representation learning model, the parameters of the GNN and the supernetwork are adjusted and optimized through the two self-supervised tasks mentioned above to obtain the trained representation learning model.

[0112] like Figure 3 As shown, in the inference phase of the representation learning model, for any first molecule, the trained GNN generates a first implicit representation of the first molecule based on its spatial structure graph and the feature representations of its components. The trained supernetwork then obtains a second implicit representation based on this first implicit representation. This second implicit representation is input into the downstream task network, which predicts the properties of the first molecule based on it.

[0113] In some embodiments, the properties of the first molecule include: the function of the first molecule, chemical reactivity, thermodynamic properties, kinetic behavior, etc. This application does not limit the specific form of the downstream task.

[0114] In some embodiments, the downstream task may be a classification task. For example, for a classification task of molecular function, the output of the downstream task network may be a discrete class label or a class probability value, which is not limited in this application.

[0115] In some embodiments, obtaining a second implicit representation through a hypernetwork based on a first implicit representation includes obtaining a second implicit representation through a hypernetwork based on a global implicit representation.

[0116] like Figure 3 As shown, in the inference phase of the representation learning model, after the trained supernetwork obtains the second implicit representation based on the first implicit representation, this second implicit representation can also be input into the INR network. The INR network outputs the attribute prediction result and the denoised first spatial coordinates based on the first implicit representation, the second implicit representation, and the first spatial coordinates. The first spatial coordinates refer to any coordinates in space, and the attribute prediction result refers to the attribute prediction result of the atom at the denoised first spatial coordinate position. Specifically: the implicit representation network determines the attribute prediction result and the denoised first spatial coordinates based on the local implicit representation, the second implicit representation, and the first spatial coordinates.

[0117] Since the first spatial coordinates of the input are arbitrary, there may be no atom at that coordinate position or the coordinate value may be deviated. Therefore, it is necessary to correct the first spatial coordinates of the input so that the corrected (denoised) spatial coordinates correspond to the position of an atom in space.

[0118] The attribute prediction result refers to the attribute prediction result of the atom at the first spatial coordinate position after denoising. In some embodiments, the first spatial coordinate after denoising is determined based on the local implicit representation, the second implicit representation, and the first spatial coordinate, and the attribute prediction result of the atom at the first spatial coordinate position is determined based on the first spatial coordinate after denoising.

[0119] For a more detailed explanation, please refer to the relevant descriptions of steps 230 to 250 above. In the above method, the second implicit representation can be provided to the downstream task network to predict the properties of the first molecule. Furthermore, the second implicit representation can also be provided to the INR network, which, based on the first spatial coordinates, can obtain the attribute prediction results and the denoised first spatial coordinates.

[0120] Furthermore, the introduction of the supernetwork enables the model to adaptively adjust when generating second implicit representations of different molecules, enhancing the model's versatility and flexibility.

[0121] This application also conducted comparative experiments with existing models for molecular function prediction tasks. The experimental results are shown in Table 1 below. The horizontal axis represents common statistical indicators used to measure the performance of prediction models: RMSE (Root Mean Square Error) is the square root of the mean of the squares of the differences between predicted and actual values. It measures the average distance between predicted and actual values; a smaller value indicates a more accurate prediction. MAE (Mean Absolute Error) is the average of the absolute values ​​of the differences between predicted and actual values. It measures the average deviation between predicted and actual values; a smaller value also indicates a more accurate prediction. SD (Standard Deviation) refers to the dispersion of predicted values ​​across a series of experiments. A smaller standard deviation indicates a more stable predicted value. R-index refers to the correlation coefficient, such as the Pearson correlation coefficient, which measures the strength and direction of the linear relationship between two variables. In prediction models, R usually represents the goodness of fit of the model, ranging from -1 to 1. A value closer to 1 indicates a better fit, and a value closer to -1 indicates a worse fit.

[0122] The vertical axis represents the various methods, which are mainly divided into ML (Machine Learning) based methods, CNN (Convolutional Neural Network) based methods, GraphDTA (Graph Neural Network for Drug-Target Interaction Prediction) methods, and GNN based methods.Specifically, these include LR (Linear Regression), SVR (Support Vector Regression), RF-Score (Random Forest Score), Pafnucy (Predicting Activity of Drugs with Machine Learning), OnionNet (Onion-Structured Neural Network), GCN (Graph Convolutional Network), GAT (Graph Attention Network), GIN (Graph Isomorphism Network), GAT-GCN (Graph Attention Network-Graph Convolutional Network), SGCN (Signed Graph Convolutional Network), GNN-DTI (Graph Neural Network for Drug-Target Interaction), DMPNN (Directed Message Passing Neural Network), MAT (Message Attentive Transformers), and DimeNet (Directional Message Passing Neural Network). Neural Network (Directional Message Passing Neural Network), CMPNN (Chemically Meaningful Pre-training Neural Network), and SIGN (Signed Graph Neural Network). LOGIP (Local & Global Implicit Pre-training) refers to the model in this application, and No-pre refers to the model in LOGIP that does not include an INR network. The data in parentheses for each metric represents uncertainty.

[0123] Table 1

[0124]

[0125] As shown in Table 1 above, the LOGIP method proposed in this application outperforms other existing models in terms of RMSE, MAE, SD, and R indicators. The method proposed in this application has been experimentally verified to have good superiority.

[0126] The technical solution provided in this application, since the learned second implicit representation is obtained through training of an implicit neural network, enables the second implicit representation learned by the supernetwork to more accurately reflect the spatial structure of the molecule, thereby improving the accuracy of molecular property prediction results in downstream tasks.

[0127] The technical solution provided in this application can be applied to scenarios involving the prediction of protein molecule properties, such as the prediction of protein molecule functions. The embodiments of this application will be specifically described below, using the aforementioned molecule as a protein molecule.

[0128] Please refer to Figure 6 This document illustrates a flowchart of a training method for a protein molecule representation learning model provided in one embodiment of this application. The execution entity for each step of this method can be a computer device; for example, the computer device could be... Figure 1 The illustrated scheme is implemented in a model training device 110 within an environment. The method may include at least one of the following steps 610 to 650.

[0129] Step 610: Obtain a spatial structure diagram of the sample protein molecule. The spatial structure diagram is used to characterize the components of the sample protein molecule and the spatial relationships between the components.

[0130] Protein molecules are stable compounds composed of atoms or amino acid residues linked by chemical bonds. Amino acid residues are the basic building blocks of protein molecules, consisting of at least two atoms. The term "amino acid residue" can be replaced with "amino acid," and this application does not limit this choice.

[0131] In some embodiments, the components of a protein molecule may include atoms and amino acid residues. Therefore, the spatial structure of a protein molecule can be described at both the atomic and amino acid residue levels; that is, the spatial structure diagram includes an atomic diagram and an amino acid diagram. In the atomic diagram, each vertex represents an atom in the sample protein molecule, and an edge connecting two vertices indicates an interaction between the corresponding atoms. Similarly, in the amino acid diagram, each vertex represents an amino acid residue in the sample protein molecule, and an edge connecting two vertices indicates an interaction between the corresponding amino acid residues.

[0132] The interaction between atoms and amino acid residues here refers to the existence of chemical bonds between atoms or amino acid residues. For example, atoms can be connected by covalent bonds, and amino acids can be connected by peptide bonds. Atoms or amino acids are linked together by chemical bonds to form a specific spatial structure.

[0133] In some embodiments, protein molecules can be broken down into multiple amino acid residues using enzymatic hydrolysis. Enzymatic hydrolysis utilizes specific enzymes (such as pepsin, trypsin, etc.) to hydrolyze protein molecules, breaking peptide bonds and thus reducing the protein molecules to their constituent amino acid residues.

[0134] The method described above describes the spatial structure of protein molecules at two different granularities. The atomic map shows the positions and interactions of atoms within the protein molecule through chemical bonds, while the amino acid map shows the spatial arrangement and interactions of amino acid residues through chemical bonds between them. This approach facilitates subsequent learning of protein molecule structural information by graph neural networks at different levels.

[0135] Step 620: Based on the spatial structure diagram and the feature representations of each component, a first implicit representation of the sample protein molecule is generated using a graph neural network. The first implicit representation is used to characterize the property features of the sample protein molecule.

[0136] In some embodiments, one-hot encoding, word embedding, or other encoding methods can be used to obtain the feature representations of atoms and amino acid residues. For example, one-hot encoding is used to encode atoms, specifically through the following process: (1) determining the type of atom, such as oxygen, nitrogen, carbon, etc.; (2) defining a unique index or number for each atom; (3) creating a zero vector with a length equal to the total number of atom types; (4) setting the element at the index position corresponding to the atom to 1, and the remaining positions to 0. The resulting vector is the one-hot encoded representation of that atom. This application does not limit the method for obtaining the feature representations of each component.

[0137] In some embodiments, for the atomic graph described above, the GNN can learn the interaction relationships between atoms and encode the learned information as an implicit representation; for the amino acid graph described above, the GNN can learn the interaction relationships between amino acid residues and encode the learned information as an implicit representation.

[0138] In some embodiments, the first implicit representation includes a global implicit representation and a local implicit representation. The global implicit representation is determined based on the characteristic representation of each of the multiple atoms of the sample protein molecule, and the local implicit representation is determined based on the characteristic representation of each of the multiple amino acid residues of the sample molecule, wherein each amino acid residue includes at least two atoms.

[0139] In some embodiments, the global implicit representation is determined based on the atomic graph, that is, the global implicit representation is generated by a graph neural network based on the atomic graph and the characteristic representation of each atom contained in the sample protein molecule.

[0140] In some embodiments, for multiple atoms contained in a protein molecule, local implicit representations of each atom are determined; a global implicit representation is generated based on the local implicit representations of each atom. The local implicit representation of an atom is determined based on that atom and other atoms connected to it.

[0141] In some embodiments, determining the local implicit representation of each of the multiple atoms contained in a protein molecule may include: for each of the multiple atoms, determining the local implicit representation of the atom using a graph neural network based on the feature representation of the atom and the feature representations of the other atoms connected to the atom.

[0142] In some embodiments, generating a global implicit representation based on the local implicit representations of the multiple atoms may include: using aggregation functions, such as pooling operations, attention mechanisms, etc., to generate a global implicit representation based on the local implicit representations of the multiple atoms.

[0143] The above method, because the global implicit representation is determined based on the characteristic representations of multiple atoms, can capture the structural characteristics of the entire protein molecule from the atomic level structural information. Since this method starts from the atomic level, the learned implicit representation can more accurately reflect the microscopic and macroscopic structural features of the protein molecule, including the spatial configuration of the protein molecule, the type and length of chemical bonds, and the relative positional relationships between atoms, providing important information support for a deeper understanding of the physicochemical properties of protein molecules.

[0144] In some embodiments, the local implicit representation is determined based on the amino acid graph, that is, the local implicit representation is generated by a graph neural network based on the amino acid graph and the characteristic representation of each amino acid residue contained in the sample protein molecule.

[0145] In some embodiments, for any one of the amino acid residues contained in a sample protein molecule, a local implicit vector of the atom group is generated based on that amino acid residue and the others connected to it.

[0146] In some embodiments, generating a local implicit vector for an atom group based on the atom group and other atom groups connected to it may include generating a local implicit representation of the atom group based on the feature representation of the atom group and the feature representations of the other atom groups connected to it. Each atom group corresponds to a local implicit representation.

[0147] The aforementioned method uses local implicit representations determined based on the characteristic representations of multiple atomic groups. These atomic groups often possess specific spatial configurations, atomic types, and reflect specific physical or chemical properties of the molecule. By focusing on the microstructure and local environment of atomic groups, this method effectively reveals local interactions and functional property changes within the molecule, facilitating subsequent predictions of atomic properties.

[0148] Step 630: The supernetwork obtains the second implicit representation based on the first implicit representation. The second implicit representation is used to provide the downstream task network to predict the properties of the protein molecule.

[0149] In some embodiments, obtaining a second implicit representation through a hypernetwork based on a first implicit representation includes obtaining a second implicit representation through a hypernetwork based on a global implicit representation.

[0150] For a detailed explanation of hypernetworks, please refer to the relevant description in step 230 above.

[0151] Downstream task networks refer to neural networks used to predict the properties of protein molecules. This network receives a second implicit representation as input and, through learning and analysis, delves into the structural and functional information of the protein molecule within this representation. It then outputs predicted molecular properties, such as function, activity, solubility, or biological activity. Protein functions can include catalyzing chemical reactions, providing structural support, transporting molecules, and cellular signaling. Because the parameters of the GNN and the weights of the supernetwork are optimized, the second implicit representation input to the downstream network can more effectively capture the structural and functional information of the protein molecule, thereby improving the accuracy of the downstream task network's predictions.

[0152] In some embodiments, this application optimizes the GNN parameters and supernet parameters by designing a self-supervised task. This self-supervised task is an atomic property prediction task, specifically steps 640 and 650 below.

[0153] Step 640: Using an implicit representation network, determine the predicted properties of atoms contained in the sample protein molecule based on the first and second implicit representations.

[0154] In this application, the properties of atoms can be the type of atoms, and the types of atoms contained in protein molecules can be hydrogen, oxygen, carbon, nitrogen, phosphorus, etc.

[0155] In some embodiments, determining the property prediction results of atoms contained in a sample protein molecule by using an implicit representation network based on a first implicit representation and a second implicit representation includes: determining the property prediction results of atoms by using an implicit representation network based on a local implicit representation, a second implicit representation, and the spatial coordinates of atoms contained in the sample protein molecule.

[0156] For more detailed information about implicit neural networks, please refer to the description in step 240 above.

[0157] Step 650: With the goal of minimizing the error of the atomic property prediction results, adjust the parameters of the protein molecule representation learning model to obtain the trained protein molecule representation learning model.

[0158] For details on how to adjust the representation learning model of protein molecules, please refer to the relevant description in step 250 above.

[0159] In some embodiments, this application also designs another self-supervised task to optimize the hypernetwork parameters. This self-supervised task is a spatial coordinate denoising task, which generates spatial coordinate prediction results of atoms through an implicit representation network based on a first implicit representation, a second implicit representation, and the spatial coordinates of atoms contained in the sample protein molecule.

[0160] In some embodiments, generating spatial coordinate prediction results of atoms through an implicit representation network based on a first implicit representation, a second implicit representation, and the spatial coordinates of atoms contained in a sample protein molecule may include: generating spatial coordinate prediction results of atoms through an implicit representation network based on a local implicit representation, a second implicit representation, and the spatial coordinates of atoms contained in a sample protein molecule.

[0161] In some embodiments, the parameters of the protein representation learning model are adjusted with the goal of minimizing the error of the spatial coordinate prediction results of atoms, resulting in a trained protein representation learning model.

[0162] The two self-supervised tasks described above, on the one hand, enable the learning of protein molecule spatial structure information at the atomic level, thereby enhancing the accuracy of the second implicit representation in expressing the protein molecule's spatial structure and further improving the prediction accuracy of downstream task networks. On the other hand, through the spatial coordinate denoising task, the model can learn stable protein molecule structures from noisy data, enhancing the model's robustness to noise and its ability to handle sparse data. Furthermore, these two self-supervised tasks can learn the spatial structure of protein molecules at the atomic level, and the second implicit representation is also provided to downstream task networks, improving the model's generalization ability in downstream task networks.

[0163] The technical solution provided in this application, on the one hand, uses a protein molecule representation learning model based on the interaction relationships between the components (such as atoms) of the protein molecule and the feature representations of each component to learn the feature representation of the protein molecule. This method avoids the problem of high computational overhead caused by using 3D mesh data of the protein molecule to learn its feature representation. On the other hand, the learning of the second implicit representation is trained through atomic-level tasks, which enables the second implicit representation learned by the supernetwork to more accurately reflect the spatial structure of the protein molecule, thereby improving the accuracy of protein molecule property prediction results in downstream tasks.

[0164] Please refer to Figure 7 This document illustrates a flowchart of a protein molecule property prediction method based on a protein molecule representation learning model, according to an embodiment of this application. The execution entity for each step of this method can be a computer device; for example, the computer device could be... Figure 1 The model in the illustrated implementation environment uses device 120. The method may include at least one of the following steps 710 to 730.

[0165] Step 710: Obtain a spatial structure diagram of the first protein molecule. The spatial structure diagram is used to characterize the components of the first protein molecule and the spatial relationships between the components.

[0166] In some embodiments, the first implicit representation includes a global implicit representation and a local implicit representation. The global implicit representation is determined based on the characteristic representation of each of a plurality of atoms of the first protein molecule, and the local implicit representation is determined based on the characteristic representation of each of a plurality of amino acid residues of the first protein molecule, wherein each amino acid residue includes at least two atoms.

[0167] In some embodiments, the spatial structure diagram includes an atomic diagram and an amino acid diagram; each vertex in the atomic diagram represents an atom in the first molecule, and an edge connecting two vertices indicates that there is an interaction between the atoms corresponding to the two vertices; each vertex in the amino acid diagram represents an amino acid residue in the first molecule, and an edge connecting two vertices indicates that there is an interaction between the amino acid residues corresponding to the two vertices.

[0168] In some embodiments, a global implicit representation is generated by a graph neural network based on the atomic diagram and the feature representations of each atom contained in the first molecule; a local implicit representation is generated by a graph neural network based on the amino acid diagram and the feature representations of each amino acid residue contained in the first molecule.

[0169] The "first protein molecule" mentioned above refers to any single protein molecule. For a more detailed explanation, please refer to the relevant descriptions of steps 210 and 610 above.

[0170] Step 720: Based on the spatial structure diagram and the feature representations of each component, a first implicit representation of the first protein molecule is generated using a graph neural network. The first implicit representation is used to characterize the property features of the first protein molecule.

[0171] For a more detailed explanation, please refer to the relevant descriptions of steps 220 and 620 above.

[0172] Step 730: The supernetwork obtains the second implicit representation based on the first implicit representation. The second implicit representation is used to provide the downstream task network to predict the properties of the first protein molecule. The parameters of the supernetwork are obtained by training the implicit representation network.

[0173] In some embodiments, the parameters of the supernetwork are obtained by training the implicit representation network, including: determining the property prediction results of atoms contained in the sample protein molecule based on the first implicit representation and the second implicit representation through the implicit representation network; adjusting the parameters of the protein molecule representation learning model (including the GNN network and the supernetwork) with the goal of minimizing the error of the atom property prediction results, to obtain the trained protein molecule representation learning model. For specific training methods, please refer to the training process described above.

[0174] In some embodiments, obtaining a second implicit representation through a hypernetwork based on a first implicit representation includes obtaining a second implicit representation through a hypernetwork based on a global implicit representation.

[0175] In some embodiments, the attribute prediction result and the denoised first spatial coordinates are determined by an implicit representation network based on the local implicit representation, the second implicit representation, and the first spatial coordinates.

[0176] For more detailed explanations, please refer to the relevant descriptions of steps 230 and 630-650 above.

[0177] The technical solution provided in this application, since the learned second implicit representation is obtained through training of an implicit neural network, enables the second implicit representation learned by the supernetwork to more accurately reflect the spatial structure of protein molecules, thereby improving the accuracy of protein molecule property prediction results in downstream tasks.

[0178] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0179] Please refer to Figure 8 This diagram illustrates a block diagram of a training apparatus for a representation learning model according to an embodiment of this application. The apparatus has the function of implementing the training method for the representation learning model described above. This function can be implemented in hardware or by hardware executing corresponding software. The apparatus 800 can be a computer device or can be installed within a computer device. The apparatus 800 may include: an acquisition module 810, a generation module 820, a obtaining module 830, a determination module 840, and an adjustment module 850.

[0180] The acquisition module 810 is used to acquire a spatial structure diagram of the sample molecule, which is used to characterize the components of the sample molecule and the spatial relationships between the components.

[0181] The generation module 820 is used to generate a first implicit representation of the sample molecule by means of the graph neural network based on the spatial structure diagram and the feature representation of each component, wherein the first implicit representation is used to characterize the attribute features of the sample molecule.

[0182] The module 830 is used to obtain a second implicit representation from the first implicit representation through the supernetwork, and the second implicit representation is used to provide the downstream task network to predict the properties of the molecule.

[0183] The determination module 840 is used to determine the property prediction results of the atoms contained in the sample molecule based on the first implicit representation and the second implicit representation through the implicit representation network.

[0184] The adjustment module 850 is used to adjust the parameters of the representation learning model with the goal of minimizing the error of the property prediction results of the atom, so as to obtain the trained representation learning model.

[0185] In some embodiments, the first implicit representation includes a global implicit representation and a local implicit representation, wherein the global implicit representation is determined based on the feature representations of multiple atoms of the sample molecule, and the local implicit representation is determined based on the feature representations of multiple groups of atoms of the sample molecule, wherein the groups of atoms include at least two atoms.

[0186] In some embodiments, the obtaining module 830 is used to obtain the second implicit representation through the supernetwork based on the global implicit representation.

[0187] In some embodiments, the determining module 840 is configured to determine the attribute prediction result of the atom by means of the implicit representation network based on the local implicit representation, the second implicit representation, and the spatial coordinates of the atoms contained in the sample molecule.

[0188] In some embodiments, the spatial structure diagram includes an atomic diagram and an atomic group diagram; each vertex in the atomic diagram represents an atom in the sample molecule, and an edge connecting two vertices indicates that there is an interaction relationship between the atoms corresponding to the two vertices; each vertex in the atomic group diagram represents a group of atoms in the sample molecule, and an edge connecting two vertices indicates that there is an interaction relationship between the groups of atoms corresponding to the two vertices.

[0189] In some embodiments, the generation module 820 is used to generate the global implicit representation using the graph neural network based on the atomic graph and the feature representations of each atom contained in the sample molecule; and to generate the local implicit representation using the graph neural network based on the atomic group graph and the feature representations of each atomic group contained in the sample molecule.

[0190] In some embodiments, the determining module 840 is used to generate a spatial coordinate prediction result of the atom through the implicit representation network based on the first implicit representation, the second implicit representation, and the spatial coordinates of the atoms contained in the sample molecule; the adjusting module 850 is used to adjust the parameters of the representation learning model with the goal of minimizing the error of the spatial coordinate prediction result of the atom, so as to obtain the trained representation learning model.

[0191] The technical solution provided in this application, on the one hand, uses a representation learning model to learn the feature representation of a molecule based on the interaction relationships between its components (such as atoms) and the feature representations of each component. This method avoids the problem of high computational overhead caused by using 3D mesh data of the molecule to learn its feature representation. On the other hand, the learning of the second implicit representation is trained through atomic-level tasks, which enables the second implicit representation learned by the supernetwork to more accurately reflect the spatial structure of the molecule, thereby improving the accuracy of molecular property prediction results in downstream tasks.

[0192] Please refer to Figure 9 This diagram illustrates a block diagram of a molecular property prediction device based on a representation learning model, according to an embodiment of this application. The device has the function of implementing the aforementioned molecular property prediction method based on a representation learning model. This function can be implemented in hardware or by hardware executing corresponding software. The device can be a computer device or can be installed within a computer device. The device 900 may include: an acquisition module 910, a generation module 920, and a result acquisition module 930.

[0193] The acquisition module 910 is used to acquire a spatial structure diagram of the first molecule, the spatial structure diagram being used to characterize the components of the first molecule and the spatial relationships between the components.

[0194] The generation module 920 is used to generate a first implicit representation of the first molecule based on the spatial structure diagram and the feature representations of each component through the graph neural network. The first implicit representation is used to characterize the attribute features of the first molecule.

[0195] The module 930 is used to obtain a second implicit representation through the supernetwork based on the first implicit representation. The second implicit representation is used to provide the second implicit representation to the downstream task network to predict the properties of the first molecule. The parameters of the supernetwork are obtained by training the implicit representation network.

[0196] In some embodiments, the first implicit representation includes a global implicit representation and a local implicit representation, wherein the global implicit representation is determined based on the characteristic representations of each of a plurality of atoms of the first molecule, and the local implicit representation is determined based on the characteristic representations of each of a plurality of groups of atoms of the first molecule, wherein the groups of atoms include at least two atoms.

[0197] In some embodiments, the apparatus 900 further includes: a determining module ( Figure 9 (Not shown in the image).

[0198] The determination module is used to determine the attribute prediction result and the denoised first spatial coordinates based on the local implicit representation, the second implicit representation, and the first spatial coordinates through the implicit representation network. The first spatial coordinates refer to any coordinates in space, and the attribute prediction result refers to the attribute prediction result of the atom at the denoised first spatial coordinate position.

[0199] The technical solution provided in this application, since the learned second implicit representation is obtained through training of an implicit neural network, enables the second implicit representation learned by the supernetwork to more accurately reflect the spatial structure of the molecule, thereby improving the accuracy of molecular property prediction results in downstream tasks.

[0200] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0201] Please refer to Figure 10 The diagram shows a structural block diagram of a computer device 1000 provided in one embodiment of this application.

[0202] Typically, computer device 1000 includes a processor 1010 and a memory 1020.

[0203] Processor 1010 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1010 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1010 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1010 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1010 may also include an AI processor for handling computational operations related to machine learning.

[0204] The memory 1020 may include one or more computer-readable storage media, which may be non-transitory. The memory 1020 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1020 are used to store a computer program configured to be executed by one or more processors to implement the above-described representation learning model training method or the above-described representation learning model-based molecular property prediction method.

[0205] Those skilled in the art will understand that Figure 10 The structure shown does not constitute a limitation on the computer device 1000, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0206] In some embodiments, a computer-readable storage medium is also provided, wherein a computer program is stored in the storage medium, the computer program being loaded and executed by a processor to implement the above-described training method for the representation learning model, or the above-described method for predicting molecular properties based on the representation learning model.

[0207] Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives), or optical disc, etc. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0208] In some embodiments, a computer program product is also provided, the computer program product comprising a computer program stored in a computer-readable storage medium, wherein a processor reads from the computer-readable storage medium and executes the computer program to implement the above-described training method for the representation learning model, or the above-described method for predicting molecular properties based on the representation learning model.

[0209] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0210] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for training a representation learning model, characterized in that, The representation learning model comprises a graph neural network, an implicit representation network and a hypernetwork, and the method comprises: obtaining a spatial structure graph of a sample molecule, the spatial structure graph being used to represent components of the sample molecule and spatial relationships between the components; generating, by the graph neural network, a first implicit representation of the sample molecule according to the spatial structure graph and feature representations of the components, the first implicit representation being used to represent attribute features of the sample molecule; obtaining, by the hypernetwork, a second implicit representation according to the first implicit representation, the second implicit representation being used to be provided to a downstream task network to predict properties of a molecule; determining, by the implicit representation network, attribute prediction results of atoms contained in the sample molecule according to the first implicit representation and the second implicit representation; adjusting parameters of the representation learning model to obtain a trained representation learning model, with the aim of minimizing errors of the attribute prediction results of the atoms.

2. The method of claim 1, wherein, The first implicit representation comprises a global implicit representation and a local implicit representation, the global implicit representation being determined based on feature representations of a plurality of atoms of the sample molecule, and the local implicit representation being determined based on feature representations of a plurality of atom groups of the sample molecule, the atom group comprising at least two atoms.

3. The method of claim 2, wherein, The obtaining, by the hypernetwork, of the second implicit representation according to the first implicit representation comprises: obtaining, by the hypernetwork, the second implicit representation according to the global implicit representation.

4. The method of claim 2, wherein, The determining, by the implicit representation network, of the attribute prediction results of the atoms contained in the sample molecule according to the first implicit representation and the second implicit representation comprises: determining, by the implicit representation network, the attribute prediction results of the atoms according to the local implicit representation, the second implicit representation and spatial coordinates of the atoms contained in the sample molecule.

5. The method of claim 2, wherein, The spatial structure graph comprises an atom graph and an atom group graph; each vertex in the atom graph represents an atom in the sample molecule, and edges between two vertices represent interaction relationships between the atoms corresponding to the two vertices; each vertex in the atom group graph represents an atom group in the sample molecule, and edges between two vertices represent interaction relationships between the atom groups corresponding to the two vertices.

6. The method of claim 5, wherein, The generating, by the graph neural network, of the first implicit representation of the sample molecule according to the spatial structure graph and the feature representations of the components comprises: generating, by the graph neural network, the global implicit representation according to the atom graph and feature representations of the atoms contained in the sample molecule; generating, by the graph neural network, the local implicit representation according to the atom group graph and feature representations of the atom groups contained in the sample molecule.

7. The method according to any one of claims 1 to 6, characterized in that, The determining, by the implicit representation network, of the attribute prediction results of the atoms contained in the sample molecule according to the first implicit representation and the second implicit representation comprises: generating, by the implicit representation network, a spatial coordinate prediction result of the atom according to the first implicit representation, the second implicit representation, and the spatial coordinates of the atoms contained in the sample molecule; adjusting parameters of the representation learning model to minimize errors of the attribute prediction result of the atom, to obtain a trained representation learning model, including: adjusting parameters of the representation learning model to minimize errors of the spatial coordinate prediction result of the atom, to obtain the trained representation learning model.

8. A method for predicting molecular properties based on a representation learning model, characterized in that, The representation learning model includes a graph neural network, an implicit representation network, and a super network, and the method includes: obtaining a spatial structure graph of a first molecule, the spatial structure graph being used to represent components of the first molecule and spatial relationships between the components; generating, by the graph neural network, a first implicit representation of the first molecule according to the spatial structure graph and feature representations of the components, the first implicit representation being used to represent attribute features of the first molecule; obtaining, by the super network, a second implicit representation according to the first implicit representation, the second implicit representation being used to provide a downstream task network to predict properties of the first molecule, and parameters of the super network being trained by the implicit representation network.

9. The method of claim 8, wherein, The first implicit representation includes a global implicit representation and a local implicit representation, the global implicit representation being determined based on feature representations of a plurality of atoms of the first molecule, and the local implicit representation being determined based on feature representations of a plurality of atom groups of the first molecule, the atom group including at least two atoms.

10. The method of claim 9, wherein, The method further includes: determining, by the implicit representation network, an attribute prediction result and a denoised first spatial coordinate according to the local implicit representation, the second implicit representation, and the first spatial coordinate, the first spatial coordinate being any coordinate in space, and the attribute prediction result being an attribute prediction result of an atom at a position of the denoised first spatial coordinate.

11. A training apparatus for representing a learning model, characterized in that, The representation learning model includes a graph neural network, an implicit representation network, and a super network, and the device includes: an obtaining module configured to obtain a spatial structure graph of a sample molecule, the spatial structure graph being used to represent components of the sample molecule and spatial relationships between the components; a generating module configured to generate, by the graph neural network, a first implicit representation of the sample molecule according to the spatial structure graph and feature representations of the components, the first implicit representation being used to represent attribute features of the sample molecule; an obtaining module configured to obtain, by the super network, a second implicit representation according to the first implicit representation, the second implicit representation being used to provide a downstream task network to predict properties of the molecule; a determining module configured to determine, by the implicit representation network, an attribute prediction result of an atom contained in the sample molecule according to the first implicit representation and the second implicit representation; an adjusting module configured to adjust parameters of the representation learning model to minimize errors of the attribute prediction result of the atom, to obtain a trained representation learning model.

12. A device for predicting a molecular property based on a representation learning model, characterized by, The representation learning model comprises a graph neural network, an implicit representation network and a hypernetwork, and the device comprises: An acquisition module is configured to acquire a spatial structure graph of a first molecule, the spatial structure graph being used to represent components of the first molecule and spatial relationships between the components; A generation module is configured to generate, by the graph neural network, a first implicit representation of the first molecule according to the spatial structure graph and feature representations of the components, the first implicit representation being used to represent attribute features of the first molecule; A obtaining module is configured to obtain, by the hypernetwork, a second implicit representation according to the first implicit representation, the second implicit representation being used to be provided to a downstream task network to predict a property of the first molecule, and parameters of the hypernetwork being obtained by training of the implicit representation network.

13. A computer device, comprising: The computer device comprises a processor and a memory, and the memory stores a computer program, which is loaded and executed by the processor to implement the method of any one of claims 1 to 7 or the method of any one of claims 8 to 10.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, which is loaded and executed by the processor to implement the method of any one of claims 1 to 7 or the method of any one of claims 8 to 10.

15. A computer program product, characterised in that, The computer program product comprises a computer program, which is loaded and executed by the processor to implement the method of any one of claims 1 to 7 or the method of any one of claims 8 to 10.

Images