Molecular image processing methods and devices based on encoders and attention mechanisms

Abstract

This application relates to the field of image processing technology and discloses a molecular image processing method and apparatus based on encoder and attention mechanism enhancement. The method includes: mapping the node feature matrix of a molecular image to a hidden feature space to generate initial node features; passing the initial node features to a single-layer encoder module; calculating attention weights through the dot product of the query and key spaces in the single-layer encoder module; updating the node features based on the calculated attention weight weight vector; extracting local neighborhood information from the initial node features through a GNN module enhanced by a multi-head attention mechanism; and generating a graph-level representation of the molecular graph based on the updated node features and the extracted local neighborhood information. This improves the ability of molecular graphs to express local information in complex structures, providing richer and more accurate data support for the overall representation of molecules.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and for example to a molecular image processing method and apparatus based on encoder and attention mechanism enhancement. Background Technology

[0002] The core task of molecular representation learning is to transform the chemical structure and physical properties of molecules into numerical representations that can be processed by computers, thereby supporting reasoning and prediction for various downstream tasks.

[0003] In recent years, with the rapid development of deep learning technology, molecular representation learning methods based on deep learning have become a research hotspot. These methods extract features directly from data through an end-to-end learning model, breaking through the dependence of traditional methods on manual rules. Compared with traditional methods, deep learning methods have shown stronger expressive power and generalization performance, especially when dealing with large-scale molecular data and complex chemical structures.

[0004] Current Transformer encoder architectures typically capture global information by stacking multiple layers of attention modules, which further exacerbates their computational complexity.

[0005] Therefore, in the AI-assisted drug discovery process, there are problems such as low computational efficiency and difficulty in adapting to large-scale molecular-level drug screening applications.

[0006] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application. Summary of the Invention

[0007] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0008] This disclosure provides a molecular image processing method based on encoder and attention mechanism enhancement. This method is applied to a neural network model including a single-layer Transformer encoder module, a multi-head attention-enhanced GNN module, and an information fusion module. The method includes:

[0009] The node feature matrix of the molecular image Mapping to the hidden feature space to generate initial node features ;

[0010] Initial node features Passed to the single-layer Transformer encoder module;

[0011] Through the query in the single-layer Transformer encoder module s and keys Attention weights are calculated using the dot product of the space.

[0012] Based on the calculated attention weight vector Update node features;

[0013] The GNN module, enhanced with a multi-head attention mechanism, extracts initial node features. Local neighborhood information;

[0014] Based on the updated node features and extracted local neighborhood information, a graph-level representation of the molecular graph is generated.

[0015] In some embodiments, the node feature matrix of the molecular image Mapping to the hidden feature space to generate initial node features It satisfies the formula:

[0016] ,

[0017] in, It is a weight matrix, where F represents the feature dimension of each node. It is the dimension of the hidden layer. It is a bias vector. It is a non-linear activation function.

[0018] In some embodiments, the initial node features are... The data passed to the single-layer Transformer encoder module includes:

[0019] Initial node features Mapping to query ,key Sum Space, satisfying the formula:

[0020] , , ,

[0021] in, , , , It is the dimension of the hidden layer.

[0022] In some embodiments, queries are performed through a single-layer Transformer encoder module. s and keys The attention weights are calculated using the dot product of the space, satisfying the formula:

[0023] ,

[0024] in, Indicator key The transpose of space, It is a normalization function.

[0025] In some embodiments, based on the calculated attention weight vector Update node features to satisfy the formula:

[0026] ,

[0027] in, It is a layer normalization operation used to enhance the stability and convergence of training.

[0028] In some embodiments, the initial node features are extracted using a GNN module enhanced by a multi-head attention mechanism. Local neighborhood information, including:

[0029] For any node and its neighboring nodes , No. The nonnormalized attention score of each attention head satisfies the formula:

[0030] ,

[0031] in, It is a weight matrix with learning capabilities. It is the first The weight vector of each attention head. This indicates a feature concatenation operation. It is a non-linear activation function. This represents the matrix transpose operation;

[0032] The calculated attention score is then passed through Function normalization yields nodes For nodes attention weights Satisfies the formula:

[0033] ,

[0034] Normalized attention weights The features of neighboring nodes are used for weighted aggregation, satisfying the formula:

[0035] ,

[0036] Among them, nodes Neighbor set Represents a node All directly connected nodes;

[0037] The final node features are obtained by weighted averaging of all attention heads, satisfying the formula:

[0038] ,

[0039] in, This represents a non-linear activation function.

[0040] In some embodiments, a graph-level representation of the molecular graph is generated based on the updated node features and extracted local neighborhood information, including:

[0041] Based on the updated node features and the extracted local neighborhood information, a weighted sum is applied and fused to obtain the fused node features, which satisfy the formula:

[0042] ,

[0043] in, This represents the molecular graph feature matrix extracted by the GNN module enhanced with multi-head attention mechanism, where each row corresponds to the final feature of a node. This is used to dynamically adjust the ratio of global to local features; This refers to the updated node features.

[0044] Based on the fused node features, a graph-level representation of the molecular graph is generated through a global average pooling operation.

[0045] This disclosure provides a data analysis apparatus based on encoder and attention mechanism enhancement, the apparatus comprising:

[0046] The processing module is used to process the node feature matrix. Mapping to the hidden feature space to generate initial node features ;

[0047] The processing module is also used to process the initial node features. Passed to the single-layer Transformer encoder module;

[0048] Single-layer Transformer encoder module for query-based s and keys Attention weights are calculated using the dot product of the space.

[0049] The single-layer Transformer encoder module is also used to calculate the attention weight vector. Update node features;

[0050] A GNN module enhanced with multi-head attention mechanism is used to extract initial node features. Local neighborhood information;

[0051] The information fusion module is used to generate corresponding image analysis representation results based on the updated node features and extracted local neighborhood information.

[0052] One embodiment provides an electronic device that includes at least one processor;

[0053] and a memory communicatively connected to the at least one processor;

[0054] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the aforementioned molecular image processing method based on encoder and attention mechanism enhancement.

[0055] This disclosure provides a storage medium storing program instructions that, when executed, perform the aforementioned molecular image processing method based on encoder and attention mechanism enhancement.

[0056] The molecular image processing method, apparatus, device, and storage medium based on encoder and attention mechanism enhancement provided in this disclosure can achieve the following technical effects:

[0057] This disclosure achieves global information propagation in molecular graphs through a self-attention mechanism in a single-layer Transformer encoder, avoiding the computational complexity caused by multi-layer stacking in traditional Transformer architectures. Simultaneously, this mechanism efficiently models long-distance dependencies between nodes in the molecular graph, significantly improving the expressive power of global molecular features. Furthermore, a multi-head graph attention network is introduced to enhance the model's ability to express local features of the molecular graph. The multi-head graph attention mechanism captures diverse interaction characteristics between nodes through parallel computation, ensuring efficient extraction and aggregation of local chemical features. This improves the ability of molecular graphs to express local information in complex structures, providing richer and more accurate data support for the overall representation of molecules.

[0058] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0059] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein:

[0060] Figure 1This is a schematic flowchart of a molecular image processing method based on encoder and attention mechanism enhancement provided in an embodiment of this disclosure;

[0061] Figure 2 This is a schematic diagram of the overall architecture of a model provided in an embodiment of this disclosure;

[0062] Figure 3 This is a molecular diagram for model input provided in an embodiment of this disclosure;

[0063] Figure 4 This is a schematic diagram of a single-layer Transformer encoder module provided in an embodiment of this disclosure;

[0064] Figure 5 This is a schematic diagram of a multi-head attention-enhanced GNN module provided in an embodiment of this disclosure;

[0065] Figure 6 This is a schematic diagram of the structure of a molecular image processing device based on encoder and attention mechanism enhancement provided in an embodiment of this disclosure;

[0066] Figure 7 This is a schematic diagram of the structure of a molecular image processing device based on encoder and attention mechanism enhancement provided in an embodiment of this disclosure. Detailed Implementation

[0067] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0068] The terms "first," "second," etc., used in the embodiments of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0069] Unless otherwise stated, the term "multiple" means two or more.

[0070] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0071] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0072] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.

[0073] The core task of molecular representation learning is to transform the chemical structure and physical properties of molecules into computer-processable numerical representations, thereby supporting inference and prediction for various downstream tasks. A key challenge lies in effectively capturing both local and global molecular characteristics while meeting the requirements of computational efficiency and generalization ability. In the early stages of molecular representation learning, traditional methods such as the Simplified Molecular Input Line Entry System (SMILES) and Extended-Connectivity Fingerprints (ECFP) were widely used. SMILES represents molecular structures in string form, which is simple in form, but its linear structure cannot reflect the complex topological relationships and spatial properties between atoms in the molecule. ECFP encodes molecular structures into fixed-length vectors through predefined rules. Although it can capture local chemical features, its rule-based design makes it difficult to flexibly adapt to diverse molecular structures and heavily relies on prior knowledge. Furthermore, these methods exhibit limited generalization ability when dealing with novel molecules unknown to the model. When faced with complex molecules and modern large-scale datasets, their expressive power and applicability gradually become insufficient, driving the exploration of more advanced data-driven representation methods.

[0074] In recent years, with the rapid development of deep learning technology, molecular representation learning methods based on deep learning have become a research hotspot. These methods extract features directly from data through an end-to-end learning model, breaking through the dependence of traditional methods on manual rules. Compared with traditional methods, deep learning methods have shown stronger expressive power and generalization performance, especially when dealing with large-scale molecular data and complex chemical structures. An important advancement is the proposal of molecular graph representation, which represents molecules as graph structures, where atoms are represented as nodes in the graph and chemical bonds as edges. In this way, molecular graphs can more naturally describe the topological properties and connectivity relationships in molecules and provide a more suitable input format for deep learning models.

[0075] Graph Neural Networks (GNNs) are a class of deep learning models that have emerged in recent years for processing graph data, achieving significant success in molecular representation learning. GNNs learn features about the relationships between nodes and their neighbors through a message-passing mechanism. Their core idea is to update the representation of the central node by aggregating features from neighboring nodes. The multi-layered stacked structure of GNNs allows them to progressively integrate local information from nodes, ultimately generating an embedded representation of the entire molecule. This approach not only flexibly adapts to graph structures of different sizes but also excels in capturing the topological properties of molecular graphs. For example, GNNs can effectively extract crucial local chemical features for predicting molecular properties while exhibiting good scalability, making them suitable for various molecular tasks.

[0076] However, despite the excellent performance of GNNs in molecular representation learning, their design has inherent limitations. Current mainstream GNN models primarily rely on local aggregation mechanisms, updating node representations by aggregating features from neighboring nodes at each layer. While this design performs well in capturing local information, it falls short in handling long-range dependencies in complex molecular graphs. Especially in molecules containing long-chain chemical structures or complex ring structures, signals between distant nodes are easily over-compressed during multi-layer propagation, leading to information loss. Another concern is the expressive power of GNNs. When dealing with molecules with similar structures but different properties, traditional GNNs exhibit certain deficiencies in discriminative ability, such as insufficient accuracy in graph isomorphism problems. This deficiency mainly stems from the limitations of their message-passing mechanism in extracting global graph information.

[0077] To overcome the limitations of GNNs, researchers have recently explored other deep learning models for processing graph-structured data. Transformer-based models, through their self-attention mechanism, can effectively capture global dependencies in graphs, compensating for GNNs' shortcomings in modeling long-range information. In molecular representation learning, the Transformer encoder models the relationships between arbitrary node pairs, generating more expressive molecular embeddings. This approach not only captures long-range dependencies in molecules but also effectively addresses the heterogeneity and complexity of molecular graphs through a flexible attention distribution mechanism.

[0078] Despite this, the Transformer model still faces the challenge of high computational cost when processing large-scale data, with its full-pair computational complexity becoming a bottleneck for resource consumption. Current Transformer encoder architectures typically capture global information through multi-layered attention modules, further exacerbating its computational complexity. Therefore, in AI-assisted drug discovery processes, the Transformer architecture needs to be optimized to improve its computational efficiency and adapt to real-world applications of large-scale drug screening.

[0079] To address the aforementioned issues, this disclosure provides a molecular image processing method, apparatus, device, and storage medium based on encoders and attention mechanisms.

[0080] The following description, in conjunction with the accompanying drawings, describes the molecular image processing method, apparatus, device, and storage medium based on encoder and attention mechanism enhancement provided in the embodiments of this disclosure.

[0081] Figure 1 This is a schematic flowchart illustrating a molecular image processing method based on encoder and attention mechanism enhancement provided in this disclosure. The method is applied to a neural network model including a single-layer Transformer encoder module, a multi-head attention-enhanced Graph Neural Network (GNN) module, and an information fusion module.

[0082] like Figure 1 As shown, molecular image processing methods enhanced by encoders and attention mechanisms can include:

[0083] S101, the node feature matrix of the molecular image Mapping to the hidden feature space to generate initial node features ;

[0084] S102, initial node features Passed to the single-layer Transformer encoder module;

[0085] S103, through the query in the single-layer Transformer encoder module s and keys Attention weights are calculated using the dot product of the space.

[0086] S104, based on the calculated attention weight vector Update node features;

[0087] S105, a GNN module enhanced with a multi-head attention mechanism, extracts initial node features. Local neighborhood information;

[0088] S106. Based on the updated node features and extracted local neighborhood information, a graph-level representation of the molecular graph is generated.

[0089] It should be noted that the molecular graph is modeled as a graph data structure. , where the set of nodes Represents atoms in a molecule, edge set This represents the chemical bonds between atoms. The model input of this invention is the node feature matrix of the molecular graph. and adjacency matrix , where the node feature matrix Each line represents the attributes of a node (atom), including the physicochemical properties of each atom in the molecule, such as atom type, number of valence electrons, etc. This indicates the number of nodes in the molecular graph. Represents the feature dimension of each node. Adjacency matrix. It is used to describe the connection relationships between atoms in a molecule.

[0090] In some embodiments, the node feature matrix of the molecular image Mapping to the hidden feature space to generate initial node features It satisfies the formula:

[0091] ,

[0092] in, It is a weight matrix, where F represents the feature dimension of each node. It is the dimension of the hidden layer. It is a bias vector. It is a non-linear activation function.

[0093] In some embodiments, the initial node features are... The data passed to the single-layer Transformer encoder module includes:

[0094] Initial node features Mapping to query ,key Sum Space, satisfying the formula:

[0095] , , ,

[0096] in, , , , It is the dimension of the hidden layer.

[0097] In some embodiments, queries are performed through a single-layer Transformer encoder module. s and keys The attention weights are calculated using the dot product of the space, satisfying the formula:

[0098] ,

[0099] in, Indicator key The transpose of space, It is a normalization function.

[0100] In some embodiments, based on the calculated attention weight vector Update node features to satisfy the formula:

[0101] ,

[0102] in, It is a layer normalization operation used to enhance the stability and convergence of training.

[0103] In some embodiments, the initial node features are extracted using a GNN module enhanced by a multi-head attention mechanism. Local neighborhood information, including:

[0104] For any node and its neighboring nodes , No. The nonnormalized attention score of each attention head satisfies the formula:

[0105] ,

[0106] in, It is a weight matrix with learning capabilities. It is the first The weight vector of each attention head. This indicates a feature concatenation operation. It is a non-linear activation function. This represents the matrix transpose operation;

[0107] The calculated attention score is then passed through Function normalization yields nodes For nodes attention weights Satisfies the formula:

[0108] ,

[0109] Normalized attention weights The features of neighboring nodes are used for weighted aggregation, satisfying the formula:

[0110] ,

[0111] Among them, nodes Neighbor set Represents a node All directly connected nodes;

[0112] The final node features are obtained by weighted averaging of all attention heads, satisfying the formula:

[0113] ,

[0114] in, This represents a non-linear activation function.

[0115] In some embodiments, a graph-level representation of the molecular graph is generated based on the updated node features and extracted local neighborhood information, including:

[0116] Based on the updated node features and the extracted local neighborhood information, a weighted sum is applied and fused to obtain the fused node features, which satisfy the formula:

[0117] ,

[0118] in, This represents the molecular graph feature matrix extracted by the GNN module enhanced with multi-head attention mechanism, where each row corresponds to the final feature of a node. This is used to dynamically adjust the ratio of global to local features; This refers to the updated node features.

[0119] Based on the fused node features, a graph-level representation of the molecular graph is generated through a global average pooling operation.

[0120] Specifically, such as Figure 2 As shown, the overall architecture of the model consists of a lightweight Transformer encoder module 201, a multi-head attention-enhanced GNN module 202, and an information fusion module 203. Figure 3 As shown, the molecular graph input to this model is processed through the node feature matrix. and adjacency matrix The representation is such that the node feature matrix describes the physicochemical properties of atoms (e.g., atom type, bond type, aromaticity, etc.), while the adjacency matrix defines the chemical bond connections between atoms. First, the node features are mapped to the latent space through a linear transformation to generate initial feature representations. These features are then passed to the global feature modeling module and the local feature extraction module.

[0121] The Transformer model, initially proposed in the field of natural language processing, effectively captures complex semantic relationships and significantly enhances the model's expressive power by stacking multiple layers of attention mechanisms. As the Transformer architecture has been increasingly applied to graph representation learning tasks, many designs have directly adopted the concept of multi-layer attention stacking to enhance the modeling ability of complex graph structures. However, in molecular graph tasks, our experimental results show that multi-layer attention stacking does not significantly improve prediction performance, and even leads to performance degradation in some cases. This may be because molecular graphs have a fixed topological structure and limited inter-node dependencies; multi-layer attention stacking easily leads to information redundancy and exacerbates gradient vanishing or degradation problems, thus negatively impacting model performance. Based on these experimental findings, this invention proposes a lightweight single-layer Transformer encoder module. Figure 4 As shown, this module employs a single-layer self-attention mechanism, which significantly reduces model complexity and computational cost while preserving the ability to model global information. Experimental results demonstrate that this module has good applicability and significant performance advantages in molecular graph tasks. Specifically, the module generates queries, keys, and values ​​through node features, and uses these mappings to calculate attention weights, thereby effectively capturing the correlations between global nodes. Subsequently, by performing a weighted summation operation on the node features, the node representation is updated, achieving efficient propagation of global information. This design not only simplifies the model architecture but also improves the efficiency and expressive power of information integration, especially performing exceptionally well when dealing with large-scale molecular datasets.

[0122] like Figure 5 As shown, the multi-head attention-enhanced GNN module extracts local features of the molecular graph in parallel through a multi-head attention mechanism. This module utilizes multiple attention heads to capture the diverse interactions between nodes, ensuring efficient extraction and representation of local chemical features. Each attention head independently calculates the importance weights of its neighboring nodes and generates local feature representations of the nodes through weighted aggregation. Subsequently, the results from multiple attention heads are combined and fused to generate more expressive local feature embeddings. The multi-head mechanism significantly enhances the model's adaptability to complex molecular structures, exhibiting particularly good robustness when handling cyclic structures and long-chain chemical structures.

[0123] The information fusion module integrates global and local features through a dynamically learned weighting mechanism to generate the final molecular graph embedding representation. Specifically, this module adaptively adjusts the contribution ratio of global and local features in the embedding representation according to task requirements, thereby ensuring the expression quality and task applicability of the molecular graph embedding. The fused molecular embedding representation is not only suitable for molecular property prediction and toxicity assessment, but can also be widely applied to various downstream tasks such as virtual drug screening.

[0124] Through the collaborative optimization design of the above modules, combined with Figure 1 Correspondingly, this invention employs a molecular image processing method enhanced by encoders and attention mechanisms. While maintaining low computational complexity, it achieves efficient global information modeling and local feature extraction for molecular graph representation. Experimental results demonstrate that this model exhibits significant performance improvements when handling complex chemical structures and large-scale molecular datasets, while maintaining extremely high computational efficiency, meeting the needs of practical drug discovery applications.

[0125] The molecular image processing method based on encoder and attention mechanism enhancement proposed in this disclosure uses a molecular graph representation learning model that can effectively capture global and local features in molecular structures. It has demonstrated high accuracy and efficiency in AI-assisted drug discovery tasks and can provide reliable technical support for related research.

[0126] In this embodiment, the global information propagation of the molecular graph is achieved through the self-attention mechanism of a single-layer Transformer encoder, avoiding the computational complexity caused by multi-layer stacking in traditional Transformer architectures. Simultaneously, this mechanism can efficiently model the dependencies between long-distance nodes in the molecular graph, thereby significantly improving the expressive power of global molecular features.

[0127] Furthermore, a multi-head graph attention network is introduced to enhance the model's ability to represent local features of the molecular graph. The multi-head graph attention mechanism captures diverse interactions between nodes through parallel computation, thereby ensuring the efficient extraction and aggregation of local chemical features. This design significantly improves the ability of the molecular graph to represent local information in complex structures, providing richer and more accurate data support for the overall representation of molecules.

[0128] An information fusion module was designed to integrate local and global feature representations. This module adaptively balances the contributions of local and global features through a weight learning mechanism, thereby generating a more accurate molecular graph embedding representation to meet the needs of downstream tasks.

[0129] and Figure 1 Corresponding to the molecular image processing method based on encoder and attention mechanism enhancement, this disclosure also provides a molecular image processing apparatus based on encoder and attention mechanism enhancement, such as... Figure 6 As shown, the device may specifically include:

[0130] Processing module 601 is used to process the node feature matrix Mapping to the hidden feature space to generate initial node features ;

[0131] Processing module 601 is also used to process the initial node features Passed to the single-layer Transformer encoder module;

[0132] Single-layer Transformer encoder module 602, used for query-based s and keys Attention weights are calculated using the dot product of the space.

[0133] The single-layer Transformer encoder module 602 is also used to calculate the attention weight vector. Update node features;

[0134] A multi-head attention mechanism-enhanced GNN module 603 is used to extract initial node features. Local neighborhood information;

[0135] The information fusion module 604 is used to generate corresponding image analysis representation results based on the updated node features and extracted local neighborhood information.

[0136] In some embodiments, the node feature matrix of the molecular image Mapping to the hidden feature space to generate initial node features It satisfies the formula:

[0137] ,

[0138] in, It is a weight matrix, where F represents the feature dimension of each node. It is the dimension of the hidden layer. It is a bias vector. It is a non-linear activation function.

[0139] In some embodiments, the initial node features are... The data passed to the single-layer Transformer encoder module includes:

[0140] Initial node features Mapping to query ,key Sum Space, satisfying the formula:

[0141] , , ,

[0142] in, , , , It is the dimension of the hidden layer.

[0143] In some embodiments, queries in a single-layer Transformer encoder module s and keys The attention weights are calculated using the dot product of the space, satisfying the formula:

[0144] ,

[0145] in, Indicator key The transpose of space, It is a normalization function.

[0146] In some embodiments, based on the calculated attention weight vector Update node features to satisfy the formula:

[0147] ,

[0148] in, It is a layer normalization operation used to enhance the stability and convergence of training.

[0149] In some embodiments, the initial node features are extracted using a GNN module enhanced by a multi-head attention mechanism. Local neighborhood information, including:

[0150] For any node and its neighboring nodes , No. The nonnormalized attention score of each attention head satisfies the formula:

[0151] ,

[0152] in, It is a weight matrix with learning capabilities. It is the first The weight vector of each attention head. This indicates a feature concatenation operation. It is a non-linear activation function. This represents the matrix transpose operation;

[0153] The calculated attention score is then passed through Function normalization yields nodes For nodes attention weights Satisfies the formula:

[0154] ,

[0155] Normalized attention weights The features of neighboring nodes are used for weighted aggregation, satisfying the formula:

[0156] ,

[0157] Among them, nodes Neighbor set Represents a node All directly connected nodes;

[0158] The final node features are obtained by weighted averaging of all attention heads, satisfying the formula:

[0159] ,

[0160] in, This represents a non-linear activation function.

[0161] In some embodiments, a graph-level representation of the molecular graph is generated based on the updated node features and extracted local neighborhood information, including:

[0162] Based on the updated node features and the extracted local neighborhood information, a weighted sum is applied and fused to obtain the fused node features, which satisfy the formula:

[0163] ,

[0164] in, This represents the molecular graph feature matrix extracted by the GNN module enhanced with multi-head attention mechanism, where each row corresponds to the final feature of a node. This is used to dynamically adjust the ratio of global to local features; This refers to the updated node features.

[0165] Based on the fused node features, a graph-level representation of the molecular graph is generated through a global average pooling operation.

[0166] Combination Figure 7 As shown in the illustration, this disclosure also provides a molecular image processing device 700 based on encoder and attention mechanism enhancement, including a processor 704 and a memory 701. Optionally, the system may further include a communication interface 702 and a bus 703. The processor 704, communication interface 702, and memory 701 can communicate with each other via the bus 703. The communication interface 702 can be used for information transmission. The processor 704 can call logical instructions in the memory 701 to execute the molecular image processing method based on encoder and attention mechanism enhancement described in the above embodiments.

[0167] Furthermore, the logic instructions in the aforementioned memory 701 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0168] The memory 701, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 704 executes functional applications and data processing by running the program instructions / modules stored in the memory 701, that is, it implements the molecular image processing method based on encoder and attention mechanism enhancement in the above embodiments.

[0169] The memory 701 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 701 may include high-speed random access memory and may also include non-volatile memory.

[0170] This disclosure provides a computer-readable storage medium storing computer-executable instructions configured as a molecular image processing method based on encoder and attention mechanism enhancement.

[0171] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0172] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of this disclosure. The aforementioned storage medium can be a non-transitory storage medium, including: a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media capable of storing program code; it can also be a transient storage medium.

[0173] The foregoing description and accompanying drawings fully illustrate embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. As used in the description of the embodiments, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used herein means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the terms “comprise” and its variations “comprises” and / or “comprising” refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Unless otherwise specified, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes the element. In this document, each embodiment may focus on describing the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, then the relevant parts can be referred to the description of the method section.

[0174] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0175] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0176] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0177] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0178] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0179] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0180] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0181] The systems and technologies described herein can be implemented in computing systems that include back-end components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0182] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0183] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0184] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A molecular image processing method based on encoder and attention mechanism enhancement, characterized in that, The method is applied to a neural network model including a single-layer Transformer encoder module, a multi-head attention-enhanced GNN module, and an information fusion module; the method includes: The node feature matrix of the molecular image Mapping to the hidden feature space to generate initial node features ; Initial node features Passed to the single-layer Transformer encoder module; Through the query in the single-layer Transformer encoder module s and keys Attention weights are calculated using the dot product of the space. Based on the calculated attention weight vector Update node features; The GNN module, enhanced with a multi-head attention mechanism, extracts initial node features. Local neighborhood information; Based on the updated node features and extracted local neighborhood information, a graph-level representation of the molecular graph is generated.

2. The method according to claim 1, characterized in that, The node feature matrix of the molecular image Mapping to the hidden feature space to generate initial node features It satisfies the formula: ， in, It is a weight matrix, where F represents the feature dimension of each node. It is the dimension of the hidden layer. It is a bias vector. It is a non-linear activation function.

3. The method according to claim 1, characterized in that, The initial node features The data passed to the single-layer Transformer encoder module includes: Initial node features Mapping to query ,key Sum Space, satisfying: , , in, , , , It is the dimension of the hidden layer.

4. The method according to claim 3, characterized in that, The query is performed through the single-layer Transformer encoder module. s and keys The attention weights are calculated using the dot product of the space, satisfying the formula: ， in, Indicator key The transpose of space, It is a normalization function.

5. The method according to claim 4, characterized in that, The calculated attention weighted vector Update node features to satisfy the formula: ， in, It is a layer normalization operation used to enhance the stability and convergence of training.

6. The method according to claim 1, characterized in that, The GNN module enhanced by the multi-head attention mechanism extracts initial node features. Local neighborhood information, including: For any node and its neighboring nodes , No. The nonnormalized attention score of each attention head satisfies the formula: ， in, It is a weight matrix with learning capabilities. It is the first The weight vector of each attention head. This indicates a feature concatenation operation. It is a non-linear activation function. This represents the matrix transpose operation; The calculated attention score is then passed through Function normalization yields nodes For nodes attention weights Satisfies the formula: ， Normalized attention weights The features of neighboring nodes are used for weighted aggregation, satisfying the formula: ， Among them, nodes Neighbor set Represents a node All directly connected nodes; The final node features are obtained by weighted averaging of all attention heads, satisfying the formula: ， in, This represents a non-linear activation function.

7. The method according to claim 1, characterized in that, The generation of a graph-level representation of the molecular graph based on the updated node features and extracted local neighborhood information includes: Based on the updated node features and the extracted local neighborhood information, a weighted sum is applied and fused to obtain the fused node features, which satisfy the formula: ， in, This represents the molecular graph feature matrix extracted by the GNN module enhanced with multi-head attention mechanism, where each row corresponds to the final feature of a node. This is used to dynamically adjust the ratio of global to local features; The updated node features; Based on the fused node features, a graph-level representation of the molecular graph is generated through a global average pooling operation.

8. A data analysis device based on encoder and attention mechanism enhancement, characterized in that, The device includes: The processing module is used to process the node feature matrix. Mapping to the hidden feature space to generate initial node features ; The processing module is also used to process the initial node features. Passed to the single-layer Transformer encoder module; Single-layer Transformer encoder module for query-based s and keys Attention weights are calculated using the dot product of the space. The single-layer Transformer encoder module is also used to calculate the attention weight vector. Update node features; A GNN module enhanced with multi-head attention mechanism is used to extract initial node features. Local neighborhood information; The information fusion module is used to generate corresponding image analysis representation results based on the updated node features and extracted local neighborhood information.

9. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory is characterized in that it stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the method according to any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

Images