Molecular property prediction method and system based on hierarchical fragment-guided cross-attention

Abstract

The application belongs to the technical field of molecular property prediction, and is especially a molecular property prediction method and system based on hierarchical fragment guided cross attention. The method comprises the following steps: S1: reading a SMILES structure string of a to-be-processed molecule; S2: obtaining a fragment set and an atom-to-fragment membership mapping; S3: projecting corresponding features into a unified hidden feature space; S5: inputting a molecule-fragment graph into a HiFraG encoder formed by stacking multiple interleaved encoding modules; S6: pre-training the encoder using unlabeled molecular data; and S7: outputting a classification value or a regression value by a task prediction network. The application improves the chemical semantic expression capability of molecular representation, enhances the information interaction capability between atomic information and fragment information, reduces the dependence on artificially labeled molecular data, and improves the molecular property prediction accuracy and generalization capability.

Description

Technical Field

[0001] This invention relates to the field of molecular property prediction technology, specifically to a method and system for predicting molecular properties based on hierarchical fragment-guided cross-attention. Background Technology

[0002] Molecular property prediction is a crucial technical step in computer-aided drug design, virtual screening, lead compound discovery, drug activity and toxicity assessment, and functional material design. By predicting the bioactivity, toxicity, solubility, lipophilicity, permeability, and other physicochemical properties of candidate molecules using computer models, a large number of candidate molecules can be screened and ranked before wet experiments, thereby narrowing down the scope of experimental validation and reducing the time and experimental costs in drug and material development.

[0003] In practical applications, high-quality molecular property tags typically require biological or chemical experiments, or costly numerical simulations. Due to limitations in experimental time, conditions, and cost, the number of molecules with reliable property tags represents only a small fraction of the explorable chemical space. In contrast, chemical databases contain a large amount of molecular data that only possesses molecular structure information but lacks target property tags. Therefore, how to utilize large-scale unlabeled molecular data to pre-learn transferable molecular representations and use limited labeled data to predict downstream molecular properties has become an important technological direction in molecular informatics and computer-aided drug design.

[0004] Molecules can be represented as graph data structures. Atoms act as graph nodes, and chemical bonds as graph edges connecting different atomic nodes. Atomic number, atomicity, formal charge, number of hydrogen atoms, hybridization type, aromaticity, ring membership, chirality, and atomic mass can serve as node features; chemical bond type, conjugation, ring membership, and stereochemistry information can serve as edge features. Graph neural networks obtain atomic node representations through message passing and feature aggregation between adjacent atoms, and generate molecular-level representations through pooling or readout operations, thereby predicting the target properties of molecules.

[0005] Existing self-supervised pre-training methods for molecular graphs mainly include node-level prediction methods and graph-level contrastive learning methods. Node-level prediction methods typically mask the properties of some atoms or chemical bonds and recover the masked properties based on the surrounding structure to learn local chemical rules. Graph-level contrastive learning methods usually construct different data augmentation views of the same molecule, improving the representational consistency between different views of the same molecule to learn global molecular features. In addition, some methods combine molecular 3D conformations, text descriptions, or physicochemical descriptors for cross-modal pre-training.

[0006] From the perspective of chemical structure, molecules possess a natural hierarchy. Multiple atoms can form functional groups, ring systems, skeletons, or synthetic fragments, and multiple chemical fragments can then combine to form a complete molecule according to specific interconnections. The biological activity and physicochemical properties of a molecule depend not only on the types of individual atoms and their local connections, but also on the types of chemical fragments, the ways in which these fragments are connected, and the overall molecular environment in which the fragments exist.

[0007] Therefore, how to reasonably represent different structural levels of molecules, such as atoms, chemical fragments, and complete molecules, and how to use large-scale unlabeled molecular data to learn molecular representations that can be transferred to downstream property prediction tasks are ongoing research directions in this field. Summary of the Invention

[0008] The purpose of this section is to outline some aspects of the embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0009] To address the aforementioned technical problems, according to one aspect of the present invention, the present invention provides the following technical solution:

[0010] A molecular property prediction method based on hierarchical fragment-guided cross-attention includes the following steps:

[0011] S1: Read the SMILES structure string of the molecule to be processed, perform a chemical validity check on the SMILES structure string, and convert it into an atomic-level molecular diagram composed of heavy atom nodes and chemical bond edges;

[0012] S2: Based on the BRICS retrosynthetic bond-breaking rules, identify cleavable chemical bonds, fragment the atomic-level molecular diagram and merge small fragments to obtain a fragment set and the membership mapping from atoms to fragments;

[0013] S3: Based on the fragment set, the connection relationship between fragments, and the membership mapping from atoms to fragments, construct a molecular-fragment graph that simultaneously contains atomic layers, fragment layers, and cross-layer edges;

[0014] S4: Generate atomic features, chemical bond features, fragment structure features, fragment functional group labels, and molecular physicochemical descriptors respectively, and project the corresponding features into a unified hidden feature space;

[0015] S5: Input the molecular-fragment graph into the HiFraG encoder, which is formed by stacking multiple interleaved coding modules. At each coding layer, atomic-level message passing, atom-to-fragment cross attention, fragment-to-atom gating fusion, fragment-level message passing, and virtual global node update are executed sequentially.

[0016] S6: The encoder is pre-trained using unlabeled molecular data, and the fragment conditional masking reconstruction loss, hierarchical cross-view contrast loss, and hierarchical chemical property prediction loss are jointly optimized.

[0017] S7: Transfer the pre-trained encoder to the target molecular property prediction task, fine-tune it using labeled molecular data, fuse atom representation, fragment representation and virtual global node representation, and output classification or regression values ​​by the task prediction network.

[0018] Molecular property prediction systems based on hierarchical fragment-guided cross-attention include:

[0019] The molecular diagram generation module is used to convert SMILES structure strings into atomic-level molecular diagrams;

[0020] The fragmentation module is used to perform BRICS key breaking, small fragment merging, and backup splitting;

[0021] The molecular-fragment graph construction module is used to generate fragment graphs and cross-layer membership edges;

[0022] The feature generation module is used to generate atomic, chemical bond, fragment, and molecular-level features.

[0023] A hierarchical coding module is used to perform multiple interleaved coding layers;

[0024] The pre-training module is used to jointly compute the three types of self-supervised losses and update the model parameters;

[0025] The fine-tuning prediction module is used to fuse multi-granularity representations and output molecular property prediction results.

[0026] Compared with the prior art, the beneficial effects of the present invention are:

[0027] Enhancing the chemical semantic representation capability of molecular representations: This invention constructs a molecular-fragment hierarchical structure based on chemical rules, enabling the model to simultaneously learn atomic-level local structural features, fragment-level functional semantic features, and molecular-level overall property features. This results in molecular representations that better conform to the laws of chemical composition, improving the model's ability to understand complex molecular structures.

[0028] Enhancing the information interaction between atomic and fragment information: Compared with existing fragment pre-training methods, this method further improves prediction performance, with an AP improvement of 13.0% for peptide function prediction and a MAE reduction of 18.2% for peptide structure prediction.

[0029] Reducing reliance on manually labeled molecular data: This invention utilizes large-scale unlabeled molecular data and jointly designs three complementary self-supervised learning tasks, enabling the model to learn effective knowledge from unlabeled molecular structures. Therefore, it can still achieve good molecular property prediction results under limited labeled data conditions.

[0030] Improving the accuracy and generalization ability of molecular property prediction: By fusing multi-granularity pre-training and hierarchical representation, this invention can effectively improve the performance of different types of molecular property prediction tasks. Experimental results show that this invention achieves better performance than existing methods in 13 benchmark tasks for molecular property prediction, with the best results achieved in 10 of them. Attached Figure Description

[0031] To more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0032] Figure 1 This is a flowchart illustrating the construction process of the molecular-fragment diagram of the present invention;

[0033] Figure 2 This is a structural diagram of the HiFraG encoder of the present invention;

[0034] Figure 3 This is a diagram illustrating the pre-training and downstream fine-tuning framework of the present invention. Detailed Implementation

[0035] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0036] Secondly, the present invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of the present invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not according to the usual scale. Furthermore, the schematic diagrams are merely examples and should not limit the scope of protection of the present invention. In addition, actual fabrication should include three-dimensional spatial dimensions of length, width, and depth.

[0037] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0038] This invention achieves full-domain reconstruction of indoor thermal environment parameters and assisted optimization of sensor deployment schemes by constructing numerical simulation samples, training multi-task neural networks, predicting virtual measurement points, and screening key measurement points, while reducing the number of actual sensors.

[0039] This invention provides a molecular property prediction method based on hierarchical fragment-guided cross-attention. Starting with molecular structure data, this method sequentially performs molecular graph generation, BRICS fragmentation, molecular-fragment graph construction, atom and fragment feature generation, hierarchical interleaved encoding, multi-granularity self-supervised pre-training, downstream task fine-tuning, and molecular property output. Each step uses the data structure generated in the previous step as input to the next, forming a complete processing flow executable by a computer. Specifically, it includes the following steps:

[0040] S1: Read the SMILES structure string of the molecule to be processed, perform a chemical validity check on the SMILES structure string, and convert it into an atomic-level molecular diagram composed of heavy atom nodes and chemical bond edges.

[0041] Atomic-level molecular diagram generation: The SMILES structure string *m* of a molecule is read, and cheminformatics tools are used to analyze the atomic, chemical bond, aromaticity, ring structure, chirality, and formal charge information. Explicit hydrogen atoms not involved in the diagram encoding of this embodiment are removed. Heavy atoms are used as nodes, and the chemical bonds between heavy atoms are retained as edges to obtain the atomic-level molecular diagram. .in, For a set of atomic nodes, This is a set of chemical bond edges. Invalid SMILES that cannot undergo valence state checking, aromatization, or structure normalization can be removed from the training corpus or marked as data awaiting manual verification.

[0042] S2, based on the BRICS retrosynthetic bond-breaking rules, identifies cleavable chemical bonds, fragments and merges small fragments of the atomic-level molecular diagram, and obtains a set of fragments and the membership mapping from atoms to fragments;

[0043] Fragmentation based on BRICS rules: such as Figure 1 As shown, BRICS fragmentation is performed on the atomic-level molecular graph. First, chemical bonds capable of retrosynthetic fragmentation are identified according to the 16 reaction environment matching rules specified by BRICS; during the identification phase, only the chemical bonds to be broken and the atomic indices of their two endpoints are recorded, without inserting dummy atoms into the molecule. Second, the identified chemical bonds to be broken are deleted from the atomic-level adjacency graph, and a breadth-first search is used to traverse the deleted adjacency graph. Each connected component is defined as an initial fragment. Simultaneously, record the set of atomic indices contained in the fragment. .

[0044] To avoid the formation of degenerate nodes due to excessively small fragment sizes, fragments with a heavy atom count less than the minimum fragment threshold are used. The initial fragments are iteratively merged, where, Initialize to 3. For a small fragment to be merged, count the number of chemical bonds that are broken or connected across fragments in the original molecular diagram between it and each adjacent fragment. Merge the small fragment into the adjacent fragment with the largest number of connections, and update the fragment atom set and fragment adjacency relationship until there are no smaller than 3. Mergeable fragments. When the number of connections is the same, the target fragment can be selected according to a preset and fixed deterministic rule to ensure consistent results when the same molecule is processed repeatedly.

[0045] When the BRICS rule does not identify a cleavable bond, a backup split is performed. The backup split selects the bond from the non-cyclic rotatable bonds that minimizes the difference in the number of heavy atoms on either side after cleavage, resulting in two relatively balanced fragments. In cases where no usable rotatable bond exists and the molecule contains a separable ring system, the split can be performed by selecting the bond connecting the ring system and the non-cyclic portion, or between different ring systems. When the molecular topology does not allow for the formation of two non-empty fragments, the complete molecule is retained as a single fragment.

[0046] S3, based on the fragment set, the connection relationship between fragments, and the membership mapping from atoms to fragments, constructs a molecular-fragment graph that simultaneously includes atomic layers, fragment layers, and cross-layer edges;

[0047] Molecular-fragment diagram generation: The final set of fragments is denoted as... And construct a fragment-level coarsening map. When at least one chemical bond exists in the original molecular diagram connecting different segments... and fragments When there are two atoms, and Establish fragment-level edges between Establish a surjective mapping from the set of atomic nodes to the set of fragment nodes. This ensures that each atomic node corresponds to a unique parent segment; thus, a cross-layer edge set is established. The final molecular-fragment diagram is obtained:

[0048]

[0049] in, Preserving the fine structure of atoms and chemical bonds, Save the fragments and their connections. Preserve the membership relationship between atoms and their parent segments. Subsequent cross-attention and gating fusion will both be based on this. The established correspondence serves as the scope for information transmission, thereby avoiding arbitrary interactions between irrelevant fragments and atoms.

[0050] S4 generates atomic features, chemical bond features, fragment structure features, fragment functional group labels, and molecular physicochemical descriptors, respectively, and projects the corresponding features into a unified hidden feature space.

[0051] Atomic characteristics and chemical bond characteristics: for each atomic node Generate 167-dimensional atomic feature vectors For each chemical bond Generate a 14-dimensional chemical bond feature vector. The feature groups are shown in the table below. Discrete variables are encoded using one-hot encoding with unknown category bits set, while continuous variables are normalized.

[0052] Table 1: Atomic Characteristics and Chemical Bond Characteristics

[0053]

[0054] Fragment structural features: For each fragment Extract 185-dimensional fragment structural features The fragment structure features are formed by sequentially piecing together three parts: the average pooling result of 167-dimensional features of all atoms within the fragment; the average pooling result of 14-dimensional features of all chemical bonds within the fragment; and a 4-dimensional topological descriptor. The 4-dimensional topological descriptor includes, in order, the normalized number of atoms, the normalized number of chemical bonds, the normalized number of rings, and the proportion of aromatic atoms. If there are no chemical bonds within the fragment, its chemical bond average pooling vector can be set to an all-zero vector.

[0055] Fragment Labels and Molecular Descriptors: To generate pre-training supervision signals, 85-dimensional multi-thermal functional group labels were calculated for each fragment using the Chem.Fragments function of RDKit. A fragment can correspond to multiple functional groups simultaneously, therefore the labels in each dimension are independent of each other. For each complete molecule, a 29-dimensional physicochemical descriptor is calculated. z-score normalization was performed using the mean and standard deviation of the pre-trained corpus. The 29-dimensional descriptors were divided into four groups: polarity / solvation, electronic, topological, and shape-related.

[0056] Table 2: Fragment Tags and Molecular Descriptors

[0057]

[0058] S5 inputs the molecular-fragment graph into the HiFraG encoder, which is formed by stacking multiple interleaved coding modules. In each coding layer, atomic-level message passing, atom-to-fragment cross attention, fragment-to-atom gating fusion, fragment-level message passing, and virtual global node update are executed sequentially.

[0059] like Figure 2As shown, the HiFraG encoder is composed of An interleaved encoding module with identical structure and learnable parameters is stacked sequentially. Here, L is 5, the unified hidden dimension d is 256, the number of heads for multi-head cross-attention H is 4, and the dropout rate is 0.1. Each interleaved encoding module must be executed in the following order: atomic update, atom-to-fragment aggregation, fragment-to-atom feedback, fragment-level update, and global node update, to ensure that each subsequent step uses the latest representation from the previous step.

[0060] 1. Input Projection: Project atomic features, chemical bond features, and fragment structure features to a unified hidden dimension. :

[0061]

[0062] in, Represents atoms The initial hidden representation after input projection Represents atoms The original feature vector, and These are learnable linear mapping matrices for atomic features and chemical bond features, respectively. Represents atoms and The edges of the chemical bonds between them are represented by projection. This represents the original eigenvector of the chemical bond. Representing fragments The initial hidden representation, Representing fragments The structural feature vector, It is a two-layer multilayer perceptron. In the pre-trained masking branch, the input representations of the masked atoms and fragments are replaced with learnable atom masking vectors, respectively. and fragment occlusion vector .

[0063] 2. Atomic-level GIN-E message passing: In the l-th coding layer, for each atom... Adjacent atoms The upper-level representation and the corresponding chemical bond representation together constitute the neighborhood message, and GIN-E updates are performed using fused edge features:

[0064]

[0065]

[0066] in, Indicates that atom a is in the first position. Candidate representations obtained by GIN-E aggregation in each coding layer For learnable scalars, For atoms The set of adjacent atoms, is the edge feature projection matrix, ReLU represents the linear rectified activation function, and MLP represents multilayer sensing; This represents the output representation of atom a in the l-th coding layer. Dropout represents random deactivation. LN represents layer normalization. Residual connections and layer normalization are used to maintain training stability.

[0067] 3. Multi-headed cross-attention from atoms to fragments: For fragments ,make This represents the set of atoms it contains. The query is the representation of the layer above the fragment, and the key and value are the corresponding atomic representations after the atomic-level update. The calculation is performed in the m-th attention head:

[0068]

[0069]

[0070] in, Indicates the first A fragment of attention. Assigned to the corresponding atom Normalized attention weights Indicates the attention head index, Indicates that it belongs to a fragment A collection of atoms , and Let represent the query, key, and value projection matrices of the m-th attention head, respectively. Representing fragments In the representation of the previous coding layer, , representing the dimension of a single attention head. Indicates the total number of attention heads. This indicates that normalization is performed on the set of atoms contained in fragment f; Representing fragments The first step after cross-attention update from atom to fragment Layer representation, The layer above it represents the multi-head attention output projection matrix. This represents the multi-head attention output projection matrix. This represents the concatenation of vectors output by each attention head. This step only calculates attention between a fragment and its corresponding atom, without constructing a dense attention matrix between all atoms and all fragments, thus maintaining the atom-fragment correspondence and reducing computational cost.

[0071] 4. Gated fusion from fragment to atom: The parent fragment representation updated with atomic information is fed back to its corresponding atom. For atoms... First obtain the parent segment M a →Represent f(a) and project it into a context vector, then calculate the dimension-wise gating coefficients:

[0072]

[0073]

[0074] in, Indicates that it is composed of atoms The parent segment represents the segment context vector obtained by projection. The learnable projection matrix representing the segment to the atom. Represents atoms The corresponding unique parent segment, This indicates that the parent segment is in the... Layer representation, Represents atoms The dimension-wise gating coefficient vector, For the Sigmoid function, This represents vector concatenation. This indicates element-wise multiplication. Gating coefficients are used to control the strength of fragment-level context received by different atoms, preventing the forced injection of fragment information with equal weight to all atoms.

[0075] 5. Fragment-level message passing: In fragment-level graphs In the middle, after completing the cross-attention from atoms to fragments This serves as the input and residual benchmark for message passing within this layer, and aggregates information from adjacent segments:

[0076]

[0077]

[0078] in, Indicates the first Learnable scalars updated at the layer fragment level. Representing fragments In a fragment-level graph, the set of adjacent fragments Representing fragments In the The layer represents the output after fragment-level message passing. Thus, the connections between adjacent chemical fragments propagate through the fragment layer, and the propagated fragment representation incorporates the latest information from its internal atoms.

[0079] 6. Virtual Global Node and Cross-Layer Aggregation: Set up a virtual global node. This is used to aggregate the global context of the atomic and fragment layers at each coding layer. After average pooling of all atomic and fragment representations at the current layer, the virtual global node is updated using two independent multilayer perceptrons:

[0080]

[0081]

[0082] in, Indicates the first Layered virtual global node representation, and Let a and f represent the atomic representation 'a' and fragment representation 'f' respectively after injecting the global context. After completing L layers of encoding, learnable weighting is applied to the atomic representations from the initial layer to the Lth layer using skip knowledge aggregation:

[0083]

[0084] in, Atoms obtained through knowledge aggregation via jumps Finally, it was stated that Indicates the total number of coding layers. Indicates the first The layer uses normalized learnable weights, with softmax representing the normalization of the weight parameters for each layer. Atomic average pooling is then used to obtain the molecular-level representation during the pre-training stage. This preserves both shallow local structural information and deep global structural information.

[0085] S6 uses unlabeled molecular data to pre-train the encoder and jointly optimizes the fragment conditional masking reconstruction loss, hierarchical cross-view contrast loss, and hierarchical chemical property prediction loss.

[0086] like Figure 3 As shown, for the same molecule in the pre-training corpus, an unmasked clean branch and a masked branch with segment conditional masking are constructed, and the two branches share the same HiFraG encoder parameters. Pre-training simultaneously performs the following three objectives.

[0087] 1. Fragment Conditional Occlusion and Attribute Reconstruction: In the t-th training round, the fragment occlusion rate is used as the basis for... Randomly select a fragment occlusion set from the fragment set. The atoms contained in the selected fragment are added to the atom masking set; and the additional atom masking rate is also applied. Masking atoms are randomly selected from the remaining atoms. Preferably, the probability of atoms within a fragment being retained in the masking set is... The value is 1.0, meaning all atoms in the selected fragment are masked. The fragment masking rate increases linearly from 0.10 to 0.35, and the additional atom masking rate increases linearly from 0.05 to 0.25.

[0088]

[0089] in, and They represent the first The fragment occlusion rate and additional atom occlusion rate for each training round. , These represent the starting and ending values ​​of the fragment occlusion rate, respectively. and Similarly, Indicates the current training round. This represents the training epoch scale corresponding to the completion of linear growth. Replace the masked atomic features with... Replace the occluded fragment features with Furthermore, the projected features of all chemical bonds with at least one endpoint belonging to the masked atom are set to zero to prevent the information of the masked region from being directly leaked through edge features. After encoding, eight atomic attribute prediction heads are used to reconstruct atom type, atomicity, formal charge, number of hydrogen atoms, hybridization type, chirality, aromaticity, and ring membership; four chemical bond attribute prediction heads are used to reconstruct bond type, conjugation, ring membership, and stereoconfiguration. Cross-entropy is used for categorical attributes, and binary cross-entropy is used for binary attributes. The preferred weights for atom reconstruction loss and chemical bond reconstruction loss are 0.8 and 0.6, respectively. The fragment conditional masking and attribute reconstruction losses are:

[0090]

[0091] in, Indicates the masked atom Attribute reconstruction loss, Indicates the number of atomic properties to be reconstructed. Indicates atomic attribute index, Indicates the first The weights of the atomic attribute loss, Indicates the relationship with the first The cross-entropy or binary cross-entropy loss function corresponding to the item attribute type. and They represent the first The predicted value of an atomic attribute versus the true label. Represents chemical bonds Attribute reconstruction loss, , , , and These represent the number of chemical bond attributes, attribute index, loss weight, predicted value, and true label, respectively.

[0092]

[0093] In the formula, This represents the total loss from fragment conditional occlusion and attribute reconstruction. Represents the set of atoms that are being hidden; This represents the set of chemical bonds that are associated with the shaded region and participate in its reconstruction. and These represent the number of elements in the respective sets; and These represent the weights of the atomic reconstruction loss and the chemical bond reconstruction loss, respectively; the summation symbol represents the summation of the reconstruction losses of each atom or chemical bond within the corresponding set.

[0094] 2. Hierarchical cross-view contrastive learning: Two layers of projection heads are used to project the atomic representation, fragment representation, and molecular representation into a 128-dimensional space, and then... Normalization. Three types of positive sample relationships are constructed: the first type is the cross-view positive sample between the masked branch atom and its parent segment in the clean branch; the second type is the same-view positive sample between the clean branch atom and its parent segment in the clean branch; the third type is the fragment-molecule positive sample between the clean branch segment and its complete molecule. Other fragments or molecules in the batch are treated as negative samples; other fragments within the same molecule that do not belong to that atom are treated as difficult negative samples, and a difficult negative sample interval, preferably 0.5, is added to their similarity logarithm.

[0095] Various contrastive losses are expressed in InfoNCE form to mask branched atoms. and clean branch parent segment For example:

[0096]

[0097] in, Represented by atoms For cross-view InfoNCE loss of anchor points, Indicates atoms in the shielded branch The representation after projection and normalization This indicates the parent segment of the atom in the clean branch. The representation after projection and normalization Indicates candidate fragments The normalized representation of the vector, where the superscript T denotes the vector transpose, and the vector dot product is used to calculate similarity. Indicates temperature parameter, Indicates the index of the candidate fragment to be compared. The term represents the hard negative sample interval, and exp and log represent the exponential and natural logarithmic functions, respectively. Among them, the non-parent segments corresponding to the same molecule... Use the hard negative sample interval; otherwise, use 0. Temperature The learnable parameter is preferably initialized to 0.07 and limited to the range of 0.01 to 1.0. The three contrast losses are obtained by weighting and summing them according to preset weights. :

[0098]

[0099] in, This represents the hierarchical cross-view contrastive learning loss. , and These represent the weights of the three types of contrast loss: cross-view atom-fragment, same-view atom-fragment, and fragment-molecule; , and These represent the corresponding single-sample contrast loss; and These represent the number of atomic nodes and the number of fragment nodes, respectively; the summation terms of each fraction represent the average loss on the corresponding node set.

[0100] 3. Hierarchical Chemical Property Prediction: At the molecular level, clean-branched molecular representations are input into four independent prediction heads, which are then regressed against standardized polarity / solvation, electronic, topological, and shape-related descriptors. Mean squared error is used to generate molecular-level descriptor loss. In the fragment layer, the occluded fragments in the occlusion branch are represented as input to the functional group prediction head, outputting 85-dimensional functional group probabilities. These probabilities are then used to calculate a binary cross-entropy with positive weights based on class frequency, along with the multi-hot functional group labels. For functional groups that appear less frequently, a larger positive sample weight is assigned to mitigate class imbalance.

[0101] Before pre-training begins Within each training epoch, a linear warm-up coefficient is used for segment loss. Preferably, Five training rounds are used. The hierarchical chemical property prediction loss is:

[0102]

[0103] in, This indicates the loss predicted by hierarchical chemical properties. This represents the regression loss of the molecular-level physicochemical descriptor. Indicates the first The segment loss warm-up coefficient for each training round. Let |F_mask| represent the set of masked segments, and |F_mask| represent the number of segments in that set. Indicates the obscured segment Functional group multi-label prediction loss.

[0104] Total pre-training loss and parameter update: The three pre-training objectives are combined into a total loss, and the parameters of the encoder, projection head, reconstruction head, and chemical property prediction head are updated through backpropagation.

[0105]

[0106] Preferably, Take 0.3, The value is set to 0.6. Pre-training can utilize the unlabeled molecular corpus of ChEMBL; the implementation uses approximately 456,000 molecules, a batch size of 256, and trains for 100 epochs using the AdamW optimizer with an initial learning rate of 5 × 10⁻⁶. -4 The weight decays to 5×10 -5 The learning rate is reduced to 1% of the initial learning rate using cosine annealing. The temperature parameter is set to 0.1 times the main learning rate without applying weight decay. The above values ​​are preferred implementation parameters and can be adjusted according to the size of the pre-training corpus, memory capacity, and target task.

[0107] S7 migrates the pre-trained encoder to the target molecule property prediction task, fine-tunes it using labeled molecule data, integrates atom representation, fragment representation and virtual global node representation, and outputs classification or regression values ​​by the task prediction network.

[0108] 1. Transferring the pre-trained encoder: After pre-training, retain the HiFraG encoder parameters and delete the atom / chemical bond reconstruction head, contrast projection head, functional group prediction head, and molecular descriptor prediction head used only for pre-training. Connect the fragment sensing readout module and the target task multilayer perceptron after the encoder. Input the SMILES structure data of the tagged molecule and generate the corresponding final atom representation according to steps S1 to S5. Fragment representation and virtual global node representation .

[0109] 2. Fragment-aware readout: First, for each atom, its representation is concatenated with the parent fragment representation, the atom attention weights are calculated, and the atom pooling representation is obtained.

[0110]

[0111] in, Represents atoms Normalized attention weights Represents atoms The final expression, This represents the final representation of the parent segment corresponding to atom 'a'. This represents vector concatenation. This represents atomic-level attention pooling. This represents a set of atomic nodes.

[0112] Secondly, set up S learnable fragment query vectors qs, and perform multi-query attention readout on the fragment representations:

[0113]

[0114]

[0115] in, This represents fragment-level multi-query pooling representation. Indicates the first The pooled result obtained from the fragment query, where S represents the number of learnable fragment queries. Indicates the first A query is assigned to a fragment. Normalized attention weights Representing fragments The final expression, Indicates the first A learnable fragment query vector, This indicates that normalization is performed on all segments. This represents the set of fragment nodes. Preferably, S is 4. Finally, the atomic pooling representation, fragment pooling representation, and virtual global node representation are concatenated, and the downstream molecular representation is obtained through a multilayer perceptron:

[0116]

[0117] 3. Fine-tuning and output: [This will be done / contained / etc.] Input a prediction head corresponding to the target task. For binary or multi-label classification tasks, the prediction head outputs one or more logarithmic values ​​for each class, and is trained using binary cross-entropy with logarithmic values; for continuous molecular property regression tasks, the prediction head outputs one or more continuous values, and is trained using mean squared error; for tasks where absolute error is the evaluation objective, L1 loss can be used. During inference, steps S1 to S5 and the above readout process are repeated for the molecules to be predicted, outputting the corresponding class probability or continuous property prediction value.

[0118] For fine-tuning, a progressive unfreezing approach is preferred: first, the pre-trained encoder is frozen and a newly initialized readout module and prediction head are trained; then, the encoder parameters are gradually released according to the training phases. The pre-trained encoder uses 0.1 times the prediction head learning rate to reduce catastrophic forgetting. In the implementation, the batch size for the MoleculeNet task is 64, training is performed for 100 epochs, and the learning rate starts from 3×10⁻⁶.-4 5×10 -4 Or 1×10 -3 Select the following: Batch size for peptide tasks is 128, training duration is 200 epochs, and learning rate is 1×10⁻⁶. -3 The preferred weight decay is 1×10. -5 And cosine annealing is used.

[0119] A molecular property prediction system based on hierarchical fragment-guided cross-attention includes at least: a molecular graph generation module for converting SMILES structure strings into atomic-level molecular graphs; a fragmentation module for performing BRICS bond breaking, small fragment merging, and backup segmentation; a molecular-fragment graph construction module for generating fragment graphs and cross-layer membership edges; a feature generation module for generating atomic, chemical bond, fragment, and molecular-level features; a hierarchical encoding module for performing multiple interleaved encoding layers; a pre-training module for jointly calculating three types of self-supervised losses and updating model parameters; and a fine-tuning prediction module for fusing multi-granularity representations and outputting molecular property prediction results.

[0120] The computer device includes a processor and memory. The memory stores the computer program and model parameters. When the processor executes the computer program, it implements steps S1 to S7. The program and parameters can also be stored in a non-transitory computer-readable storage medium, or deployed on a server, workstation, graphics processing unit cluster, or cloud computing platform. The training and prediction phases can be executed on the same device or deployed separately.

[0121] Optional Implementation Methods and Technical Boundaries: Without altering the core processing relationship of hierarchical representation of atoms and fragments and layer-by-layer bidirectional interaction, the atom-level or fragment-level message passing network can employ GIN, GCN, GraphSAGE, or graph attention networks; the number of encoding layers, hidden dimensions, number of attention heads, occlusion ratio, and loss weights can be adjusted according to the data scale; the molecular-level downstream task can be a classification task, a regression task, or a multi-task prediction task. All the above substitutions should maintain the structural mapping from atoms to parent fragments, and within the same encoding layer, the fragment should be updated using atoms first, and then the updated fragment should be used to feed back the atoms. This implementation uses a two-dimensional molecular structure as input and does not require prior knowledge of the three-dimensional equilibrium conformation of the molecule.

[0122] Example:

[0123] Example 1: Constructing a HiFraG pre-trained model based on unlabeled molecular corpus.

[0124] System Composition: This embodiment employs a computer device including a processor, memory, and an optional graphics processor. The memory contains a molecular structure analysis program, an RDKit chemical information processing component, a graph neural network training program, and model parameter files. The training data consists of approximately 456,000 unlabeled molecules obtained from the ChEMBL database and processed, with each molecule represented by a SMILES structure string.

[0125] Molecular-fragment graph construction: A chemivalidity check is performed on each SMILES to generate a heavy atom-bond graph. Based on the 16 types of retrosynthetic reaction environments of BRICS, cleavable bonds are identified. After deleting these bonds, a breadth-first search is used to obtain connected components. Fractions with fewer than 3 heavy atoms are iteratively merged into the adjacent fragment with the most connections. When no BRICS cleavable bonds exist, the backup segmentation rules in Part IV are applied. Subsequently, a fragment-level graph and a cross-layer mapping from each atom to a unique parent fragment are constructed to form the molecular-fragment graph.

[0126] Feature composition: Each atom uses 167 features, including atomic number, atomicity, formal charge, number of hydrogen atoms, hybridization type, aromaticity, ring membership, chirality, and normalized atomic mass; each chemical bond uses 14 features, including bond type, conjugation, ring membership, and stereochemistry. Each fragment uses 185 initial structural features, formed by concatenating four topological descriptors: the average value of atomic features within the fragment, the average value of chemical bond features within the fragment, and the normalized number of atoms, normalized number of chemical bonds, normalized number of rings, and proportion of aromatic atoms. Simultaneously, 85-dimensional multi-thermal tags for fragment functional groups are generated using RDKitChem.Fragments, and a 29-dimensional molecular physicochemical descriptor is calculated.

[0127] Encoder structure: Atom, chemical bond, and fragment features are projected into a 256-dimensional hidden space, and five interleaved encoding modules are stacked sequentially. Each module includes atomic-level GIN-E message passing, four-head atom-to-fragment cross-attention, fragment-to-atom gated fusion, fragment-level GIN message passing, and virtual global node updates. After encoding, cross-layer skip knowledge aggregation is performed. The network dropout rate is set to 0.1.

[0128] Pre-training process: Clean branches and masked branches are constructed for the same molecule. In fragment conditional masking, the fragment masking rate linearly increases from 0.10 to 0.35 within 100 training epochs, and the additional atom masking rate linearly increases from 0.05 to 0.25. All atoms within the selected fragment are added to the masking set, and the chemical bond features connected to the masked atoms are set to zero. The model reconstructs 8 sets of atomic properties and 4 sets of chemical bond properties. Hierarchical cross-view contrastive learning uses 128-dimensional projection, with an initial temperature of 0.07 and a limit of 0.01–1.0. Non-parent fragments within the same molecule use a difficult negative sample interval of 0.5. Hierarchical chemical property prediction simultaneously regresses the 29-dimensional molecular descriptor and predicts 85-dimensional fragment functional group labels. Fragment loss is linearly warmed up in the first 5 training epochs. The weights for HCCL and HCPP losses are 0.3 and 0.6, respectively, and the weights for atomic and chemical bond reconstruction in FCM are 0.8 and 0.6, respectively.

[0129] Training conditions: AdamW optimizer was used, batch size was 256, training was conducted for 100 epochs, and the initial learning rate was 5×10. -4 The weight decays to 5×10 -5 The learning rate was reduced to 1% of the initial learning rate using cosine annealing. The temperature parameter was set to 0.1 times the main learning rate without applying weight decay. After training, the parameters of the HiFraG encoder were saved for fine-tuning in subsequent classification, regression, or peptide mapping tasks.

[0130] Effect: The above process yields a pre-trained encoder that simultaneously incorporates atomic local environment, fragment structural semantics, and molecular global information. Taking the average ROC-AUC of eight MoleculeNet classification tasks as an example, the complete model achieves 80.98%.

[0131] Example 2: Small molecule classification prediction using a pre-trained model.

[0132] Data composition: Eight classification datasets from MoleculeNet were selected: BBBP2039 molecules, Tox217831 molecules, ToxCast8575 molecules, SIDER1427 molecules, ClinTox1478 molecules, MUV93087 molecules, HIV41127 molecules, and BACE1513 molecules. Each dataset was divided into training, validation, and test sets according to the molecular skeleton in a ratio of 8:1:1, and experiments were repeated using three random seeds. ROC-AUC was used as the evaluation metric.

[0133] Model Construction and Fine-tuning: The HiFraG encoder obtained in Example 1 is loaded, and the reconstruction head, projection head, functional group prediction head, and molecular descriptor prediction head used in the pre-training stage are removed. The fragment awareness readout module and the task prediction network are then connected. The fragment awareness readout module calculates atomic attention weights based on the atomic representation and the parent fragment representation, and sets four learnable fragment query vectors to aggregate the fragment representation. Then, the atomic pooling representation, fragment pooling representation, and virtual global node representation are concatenated and input into the multilayer perceptron. The classification task uses a binary cross-entropy loss with logarithmic values.

[0134] Execution conditions: Batch size set to 64, training for 100 epochs, weight decay of 1×102 -5 The learning rate is 3×10 -5 5×10 -4 and 1×10 -3 The dropout rate was selected based on the validation set, with values ​​ranging from 0.10, 0.15, and 0.20, and cosine annealing was used. First, the newly initialized readout module and prediction head were trained, followed by a gradual release of the pre-trained encoder parameters; the encoder learning rate was set to 0.1 times the prediction head learning rate.

[0135] Results: As shown in Table 3, the average ROC-AUC of this invention on eight MoleculeNet classification benchmarks is 80.98%. Compared with some representative methods, this embodiment achieves the highest results on six of the classification datasets, with an average ROC-AUC 2.78 percentage points higher than the strongest comparison method, indicating that the pre-trained encoder can transfer to various physiological, toxicological, and biological activity classification tasks with a limited number of labeled molecules. Results are reported as mean ± standard deviation. The rightmost column shows the average ROC-AUC for all eight benchmarks.

[0136] Table 3: ROC-AUC (%) performance of different models on eight MoleculeNet classification benchmarks

[0137]

[0138] Example 3: Regression of continuous properties of small molecules using a pre-trained model.

[0139] Data composition: Three regression datasets from MoleculeNet were selected: the ESOL water solubility dataset containing 1128 molecules, the FreeSolv hydration free energy dataset containing 642 molecules, and the Lipophilicity dataset containing 4200 molecules. Each dataset was partitioned using a molecular backbone and three random seeds. The ratio of training set, validation set, and test set was 8:1:1. The prediction results were evaluated using the root mean square error (RMSE), with a lower value indicating a smaller prediction error.

[0140] Model Structure and Execution Process: The same pre-trained encoder and fragment-aware readout module as in Example 2 are used. The classification prediction head is replaced with a regression prediction head that outputs a single continuous value, and mean squared error is used as the training loss. Fine-tuning of batch size, training epochs, candidate learning rate, weight decay, cosine annealing, and progressive unfreezing methods are the same as in Example 2. During inference, the SMILES of the molecule to be tested are sequentially converted into an atomic-level molecular map, a BRICS fragment-level map, and a cross-layer mapping. The molecular representation is obtained through the HiFraG encoder and fragment-aware readout module, and then the corresponding continuous property values ​​are output.

[0141] Effects: As shown in Table 4, the experimental results show that the RMSE values ​​of this embodiment on ESOL, FreeSolv, and Lipophilicity are 0.739±0.020, 1.768±0.136, and 0.680±0.021, respectively. Compared with some representative methods, ESOL and Lipophilicity achieved the lowest RMSE, while FreeSolv achieved the second lowest RMSE. This result indicates that the atom-fragment hierarchical representation can be used for regression prediction of different continuous physicochemical properties such as solubility, hydration free energy, and lipophilicity.

[0142] Results are expressed as mean ± standard deviation.

[0143] Table 4: RMSE performance of different models on three MoleculeNet regression benchmarks

[0144]

[0145] Example 4: Predicting long-range peptide graph properties using a pre-trained model.

[0146] Data Composition: The Peptides-func and Peptides-struct datasets from the Long-Range Graph Benchmark were selected. Both datasets contain 15,535 peptide maps. Peptides-func includes 10 multi-label peptide function classification tasks to predict antibacterial, antiviral, and anticancer functions, with mean precision (AP) as the evaluation metric. Peptides-struct includes 11 regression tasks related to three-dimensional structural properties, with mean absolute error (MAE) as the evaluation metric. Data partitioning follows the official stratified random partitioning specified by LRGB.

[0147] Model Structure and Execution Process: The encoder obtained in Example 1 is used to process atoms, chemical bonds, and BRICS fragments in the peptide graph. Information across distant chemical regions is propagated through layer-by-layer atom-fragment bidirectional interaction, and a peptide graph-level representation is formed through four fragment query vectors and virtual global nodes. Peptides-func uses multi-label binary cross-entropy loss, and Peptides-struct uses L1 loss. The fine-tuning batch size is 128, training is conducted for 200 epochs, and the learning rate is 1×10⁻⁶. -3 The weight decays to 1×10 -5 The dropout rate is 0.1, the encoder learning rate is 0.1 times the master learning rate, and cosine annealing and progressive unfreezing are employed.

[0148] Effects: As shown in Table 5, the experimental results show that the AP of this embodiment on Peptides-func is 0.703±0.002, and the MAE on Peptides-struct is 0.256±0.003, both achieving optimal results compared to the comparative methods. Compared with GraphFP, which uses fragment information but lacks equivalent layer-by-layer bidirectional interactions during encoding, Peptides-func improves the AP by 13.0%, and Peptides-struct reduces the MAE by 18.2%. These results demonstrate that this invention is not only applicable to conventional small molecule graphs but can also handle peptide molecule graphs with a large number of nodes and long-range structure dependence.

[0149] Table 5: Performance of different models in LRGB peptide benchmark tests

[0150]

[0151] Peptides-func uses AP, and Peptides-struct uses MAE. Higher AP and lower MAE indicate better performance.

[0152] Although the present invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the disclosed embodiments can be combined with each other in any manner. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A molecular property prediction method based on hierarchical fragment-guided cross-attention, characterized in that, Includes the following steps: S1: Read the SMILES structure string of the molecule to be processed, perform a chemical validity check on the SMILES structure string, and convert it into an atomic-level molecular diagram composed of heavy atom nodes and chemical bond edges; S2: Based on the BRICS retrosynthetic bond-breaking rules, identify cleavable chemical bonds, fragment the atomic-level molecular diagram and merge small fragments to obtain a fragment set and the membership mapping from atoms to fragments; S3: Based on the fragment set, the connection relationship between fragments, and the membership mapping from atoms to fragments, construct a molecular-fragment graph that simultaneously contains atomic layers, fragment layers, and cross-layer edges; S4: Generate atomic features, chemical bond features, fragment structure features, fragment functional group labels, and molecular physicochemical descriptors respectively, and project the corresponding features into a unified hidden feature space; S5: Input the molecular-fragment graph into the HiFraG encoder, which is formed by stacking multiple interleaved coding modules. At each coding layer, atomic-level message passing, atom-to-fragment cross attention, fragment-to-atom gating fusion, fragment-level message passing, and virtual global node update are executed sequentially. S6: The encoder is pre-trained using unlabeled molecular data, and the fragment conditional masking reconstruction loss, hierarchical cross-view contrast loss, and hierarchical chemical property prediction loss are jointly optimized. S7: Transfer the pre-trained encoder to the target molecular property prediction task, fine-tune it using labeled molecular data, fuse atom representation, fragment representation and virtual global node representation, and output classification or regression values ​​by the task prediction network.

2. The molecular property prediction method based on hierarchical fragment-guided cross-attention according to claim 1, characterized in that, In S1, the method for converting the atomic-level molecular diagram is as follows: The SMILES structure string m of a molecule is read, and the atomic, chemical bond, aromaticity, ring structure, chirality, and formal charge information are analyzed using cheminformatics tools. Explicit hydrogen atoms that do not participate in the diagram encoding of this embodiment are removed. Heavy atoms are used as nodes, and the chemical bonds between heavy atoms are retained as edges to obtain the atomic-level molecular diagram. ,in, For a set of atomic nodes, This is a set of chemical bond edges; invalid SMILES that cannot complete valence state checking, aromatization, or structure normalization are removed from the training corpus or marked as data to be manually verified.

3. The molecular property prediction method based on hierarchical fragment-guided cross-attention according to claim 1, characterized in that, The specific method of S2 is as follows: Chemical bonds capable of retrosynthetic cleavage are identified according to the 16 types of reaction environment matching rules specified by BRICS; during the identification stage, only the chemical bonds to be cleaved and the atomic indices of their two endpoints are recorded, without inserting virtual atoms into the molecule; the identified chemical bonds to be cleaved are deleted from the atomic-level adjacency graph, and a breadth-first search is used to traverse the deleted adjacency graph; each connected component is defined as an initial fragment. Simultaneously, record the set of atomic indices contained in the fragment. ; For heavy atoms with a number less than the minimum fragment threshold The initial fragments are iteratively merged, where, Initialize to 3; for a small fragment to be merged, count the number of chemical bonds that are broken or connected across fragments in the original molecular diagram between it and each adjacent fragment, merge the small fragment into the adjacent fragment with the largest number of connections, and update the fragment atom set and fragment adjacency relationship until there are no smaller than 3. Mergeable fragments; when the number of connections is the same, the target fragment is selected according to a preset and fixed deterministic rule to ensure consistent results when the same molecule is processed repeatedly; When the BRICS rule does not identify a cleavable chemical bond, a backup split is performed. The backup split selects the chemical bond with the smallest difference in the number of heavy atoms on both sides after the split from the non-cyclic rotatable bond to obtain two relatively balanced fragments. In the case where there are no usable rotatable bonds and the molecule contains separable ring systems, the split is performed by the connecting bond between the ring system and the non-cyclic part or between different ring systems. When the molecular topology does not allow the formation of two non-empty fragments, the complete molecule is retained as a single fragment.

4. The molecular property prediction method based on hierarchical fragment-guided cross-attention according to claim 1, characterized in that, In S3, the method for constructing the molecular-fragment diagram is as follows: the final fragment set is denoted as... And construct a fragment-level coarsening map. ; When the original molecular diagram contains at least one chemical bond connecting different segments and fragments When there are two atoms, and Establish fragment-level edges between Establish a surjective mapping from the set of atomic nodes to the set of fragment nodes. This ensures that each atomic node corresponds to a unique parent segment; thus, a cross-layer edge set is established. The final molecular-fragment diagram is obtained: in, Preserving the fine structure of atoms and chemical bonds, Save the fragments and their connections. Preserve the membership relationship between atoms and their parent segments; subsequent cross-attention and gating fusion are both based on this. The established correspondence serves as the scope of information transmission.

5. The molecular property prediction method based on hierarchical fragment-guided cross-attention according to claim 1, characterized in that, In S4, for each atomic node Generate 167-dimensional atomic feature vectors For each chemical bond Generate a 14-dimensional chemical bond feature vector. ; Discrete variables are encoded using one-hot encoding and an unknown category bit is set, while continuous variables are normalized. For each segment Extract 185-dimensional fragment structural features The fragment structure features are formed by sequentially splicing three parts: the average pooling result of 167-dimensional features of all atoms within the fragment; the average pooling result of 14-dimensional features of all chemical bonds within the fragment; and a 4-dimensional topological descriptor; the 4-dimensional topological descriptor includes, in order, the normalized number of atoms, the normalized number of chemical bonds, the normalized number of rings, and the proportion of aromatic atoms; if there are no chemical bonds within the fragment, its chemical bond average pooling vector is set to an all-zero vector; 85-dimensional multi-functional group labels were calculated for each fragment using the Chem.Fragments function of RDKit. A single fragment corresponds to multiple functional groups, therefore the labels in each dimension are independent of each other; for each complete molecule, a 29-dimensional physicochemical descriptor is calculated. z-score normalization was performed using the mean and standard deviation of the pre-trained corpus; the 29-dimensional descriptors were divided into four groups: polarity / solvation, electronic, topological, and shape-related.

6. The molecular property prediction method based on hierarchical fragment-guided cross-attention according to claim 1, characterized in that, In S5, the HiFraG encoder is composed of... Interleaved encoding modules with identical structures and learnable parameters are stacked sequentially; where L is 5, the unified hidden dimension d is 256, the number of heads for multi-head cross-attention H is 4, and the dropout rate is 0.1; each interleaved encoding module must be executed in the following order: atomic-level update, atom-to-fragment aggregation, fragment-to-atom feedback, fragment-level update, and global node update; the specific method is as follows: Input projection: Projecting atomic features, chemical bond features, and fragment structure features to a unified hidden dimension. : in, Represents atoms The initial hidden representation after input projection Represents atoms The original feature vector, and These are learnable linear mapping matrices for atomic features and chemical bond features, respectively. Represents atoms and The edges of the chemical bonds between them are represented by projection. This represents the original eigenvector of the chemical bond. Representing fragments The initial hidden representation, Representing fragments The structural feature vector, It is a two-layer multilayer perceptron; in the pre-trained masking branch, the input representations of the masked atoms and fragments are replaced with learnable atom masking vectors, respectively. and fragment occlusion vector ; Atomic-level GIN-E message passing: In the l-th coding layer, for each atom Adjacent atoms The upper-level representation and the corresponding chemical bond representation together constitute the neighborhood message, and GIN-E updates are performed using fused edge features: in, Indicates that atom a is in the first position. Candidate representations obtained by GIN-E aggregation in each coding layer For learnable scalars, For atoms The set of adjacent atoms, is the edge feature projection matrix, ReLU represents the linear rectified activation function, and MLP represents multilayer sensing; This represents the output representation of atom a in the l-th coding layer. Dropout represents random deactivation. LN represents layer normalization. Residual connections and layer normalization are used to maintain training stability. The symbol "←" indicates that the variable on the left side of the equation is updated with the result on the right side of the equation. Multi-head cross-attention from atoms to fragments: for fragments ,make This represents the set of atoms it contains; calculated in the m-th attention head: in, Indicates the first A fragment of attention. Assigned to the corresponding atom Normalized attention weights Indicates the attention head index, Indicates that it belongs to a fragment A collection of atoms , and Let represent the query, key, and value projection matrices of the m-th attention head, respectively. Representing fragments In the representation of the previous coding layer, , representing the dimension of a single attention head. Indicates the total number of attention heads. This indicates that normalization is performed on the set of atoms contained in fragment f; Representing fragments The first step after cross-attention update from atom to fragment Layer representation, The layer above it represents the multi-head attention output projection matrix. This represents the multi-head attention output projection matrix. This represents the concatenation of vectors output by each attention head; this step only calculates attention between a fragment and its corresponding atom, and does not construct a dense attention matrix between all atoms and all fragments; Fragment-to-Atom Gated Fusion: The parent fragment representation updated with atomic information is fed back to its corresponding atom. First obtain the parent segment M a →Represent f(a) and project it into a context vector, then calculate the dimension-wise gating coefficients: in, Indicates that it is composed of atoms The parent segment represents the segment context vector obtained by projection. The learnable projection matrix representing the segment to the atom. Represents atoms The corresponding unique parent segment, This indicates that the parent segment is in the... Layer representation, Represents atoms The dimension-wise gating coefficient vector, For the Sigmoid function, This represents vector concatenation. This indicates element-wise multiplication, and the gating coefficients are used to control the strength of the segment-level context received by different atoms; Fragment-level message passing: in fragment-level graphs In the middle, after completing the cross-attention from atoms to fragments This serves as the input and residual benchmark for message passing within this layer, and aggregates information from adjacent segments: in, Indicates the first Learnable scalars updated at the layer fragment level. Representing fragments In a fragment-level graph, the set of adjacent fragments Representing fragments In the The output representation after the layer completes fragment-level message passing; thus, the connection relationship between adjacent chemical fragments propagates in the fragment layer, and the propagated fragment representation has absorbed the latest information of its internal atoms; Virtual Global Nodes and Cross-Layer Aggregation: Setting up a virtual global node This is used to aggregate the global context of the atomic and fragment layers at each coding layer; after average pooling of all atomic and fragment representations at this layer, the virtual global node is updated using two independent multilayer perceptrons: in, Indicates the first Layered virtual global node representation, and Let a and f represent the atomic representation 'a' and fragment representation 'f' respectively after injecting global context; after completing L-layer encoding, learnable weighting is applied to the atomic representations from the initial layer to the L-th layer using skip knowledge aggregation: in, Atoms obtained through knowledge aggregation via jumps Finally, it was stated that Indicates the total number of coding layers. Indicates the first The layer uses normalized learnable weights, with softmax representing the normalization of the weight parameters for each layer; and molecular-level representations are obtained during the pre-training stage through atomic representation average pooling. , It represents a set of atomic nodes, thereby preserving both shallow local structural information and deep global structural information.

7. The molecular property prediction method based on hierarchical fragment-guided cross-attention according to claim 1, characterized in that, In S6, for the same molecule in the pre-training corpus, an unmasked clean branch and a masked branch that performs fragment conditional masking are constructed. The two branches share the same HiFraG encoder parameters. The pre-training simultaneously performs the following three objectives: (1) Fragment conditional occlusion and attribute reconstruction: In the t-th training round, the fragment occlusion rate is used as the basis for the reconstruction. Randomly select a fragment occlusion set from the fragment set. The atoms contained in the selected fragment are added to the atom masking set; and the additional atom masking rate is also applied. The probability of randomly selecting additional masking atoms from the remaining atoms and retaining atoms within the fragment in the masking set. The value is 1.0, meaning all atoms of the selected fragment are masked. The fragment masking rate increases linearly from 0.10 to 0.35, and the additional atom masking rate increases linearly from 0.05 to 0.

25. in, and They represent the first The fragment occlusion rate and additional atom occlusion rate for each training round. , These represent the starting and ending values ​​of the fragment occlusion rate, respectively. and Similarly, Indicates the current training round. This represents the training epoch scale corresponding to the completion of linear growth; the masked atomic features are replaced with... Replace the occluded fragment features with The projected features of all chemical bonds with at least one endpoint belonging to the masked atom are set to zero. After encoding, eight atomic attribute prediction heads are used to reconstruct atom type, atomicity, formal charge, number of hydrogen atoms, hybridization type, chirality, aromaticity, and ring membership. Four chemical bond attribute prediction heads are used to reconstruct bond type, conjugation, ring membership, and stereoconfiguration. Cross-entropy is used for categorical attributes, and binary cross-entropy is used for binary attributes. The weights of atom reconstruction loss and chemical bond reconstruction loss are 0.8 and 0.6, respectively. The fragment conditional masking and attribute reconstruction losses are: in, Indicates the masked atom Attribute reconstruction loss, Indicates the number of atomic properties to be reconstructed. Indicates atomic attribute index, Indicates the first The weights of the atomic attribute loss, Indicates the relationship with the first The cross-entropy or binary cross-entropy loss function corresponding to the item attribute type. and They represent the first The predicted value of an atomic attribute versus the true label. Represents chemical bonds Attribute reconstruction loss, , , , and These represent the number of chemical bond attributes, attribute index, loss weight, predicted value, and true label, respectively. in, This represents the total loss from fragment conditional occlusion and attribute reconstruction. Represents the set of atoms that are being hidden; This represents the set of chemical bonds that are associated with the shaded region and participate in its reconstruction. and These represent the number of elements in the respective sets; and These represent the weights of the atomic reconstruction loss and the chemical bond reconstruction loss, respectively; the summation symbol represents the summation of the reconstruction losses of each atom or chemical bond within the corresponding set; (2) Hierarchical cross-view contrastive learning: Atom representation, fragment representation, and molecular representation are projected onto 128-dimensional space using two layers of projection heads respectively, and then... Normalization; construct three types of positive sample relationships: the first type is the cross-view positive sample between the masked branch atom and its parent segment in the clean branch; the second type is the same-view positive sample between the clean branch atom and its parent segment in the clean branch; the third type is the segment-molecule positive sample between the clean branch segment and its complete molecule; other segments or molecules in the batch are used as negative samples; other segments within the same molecule that do not belong to that atom are used as difficult negative samples, and a difficult negative sample interval of preferably 0.5 is added to their similarity logarithm. Various contrastive losses are implemented using the InfoNCE form to mask branched atoms. and clean branch parent segment For example: in, Represented by atoms For cross-view InfoNCE loss of anchor points, Indicates atoms in the shielded branch The representation after projection and normalization This indicates the parent segment of the atom in the clean branch. The representation after projection and normalization Indicates candidate fragments The normalized representation of the vector, where the superscript T denotes the vector transpose, and the vector dot product is used to calculate similarity. Indicates temperature parameter, Indicates the index of the candidate fragment to be compared. The term represents the hard negative sample interval, and exp and log represent the exponential function and the natural logarithm function, respectively; where, the term corresponds to the non-parent segment of the same molecule. Use the hard negative sample interval, otherwise use 0; temperature For learnable parameters, the initial value is preferably 0.07, and limited to between 0.01 and 1.

0. The three types of contrast losses are obtained by weighting and summing them according to preset weights. : in, This represents the hierarchical cross-view contrastive learning loss. , and These represent the weights of the three contrast losses: cross-view atom-fragment, same-view atom-fragment, and fragment-molecule, respectively. , and These represent the corresponding single-sample contrast loss. and These represent the number of atomic nodes and the number of fragment nodes, respectively, and the summation of each fraction represents the average loss on the corresponding node set; (3) Hierarchical chemical property prediction: At the molecular level, the clean branched molecular representation is input into four independent prediction heads, which are then regressed against the standardized polarity / solvation, electronic, topological, and shape-related descriptors. The mean squared error is used to form the molecular-level descriptor loss. At the fragment layer, the occluded fragments in the occlusion branch are input into the functional group prediction head, outputting 85-dimensional functional group probabilities. These probabilities are then compared with the multi-hot functional group labels to calculate a binary cross-entropy with positive weights based on class frequency. For functional groups that appear less frequently, assign larger positive sample weights; before pre-training begins... Within each training epoch, a linear warm-up coefficient is used for segment loss. , Taking 5 training rounds, the hierarchical chemical property prediction loss is: in, This indicates the loss predicted by hierarchical chemical properties. This represents the regression loss of the molecular-level physicochemical descriptor. Indicates the first The segment loss warm-up coefficient for each training round. Let |F_mask| represent the set of masked segments, and |F_mask| represent the number of segments in that set. Indicates the obscured segment Functional group multi-label prediction loss; The three pre-trained objectives are combined into a total loss, and the parameters of the encoder, projection head, reconstruction head, and chemical property prediction head are updated through backpropagation: in, This represents the total pre-training loss. and These represent the weights of the contrastive learning loss and the chemical property prediction loss in the total loss, respectively.

8. The molecular property prediction method based on hierarchical fragment-guided cross-attention according to claim 1, characterized in that, The specific method of S7 is as follows: Transferring the pre-trained encoder: After pre-training, retain the HiFraG encoder parameters and remove the atom / chemical bond reconstruction head, contrast projection head, functional group prediction head, and molecular descriptor prediction head used only for pre-training. Connect the fragment sensing readout module and the target task multilayer perceptron after the encoder; input the SMILES structure data of tagged molecules and generate the corresponding final atom representations according to S1 to S5. Fragment representation and virtual global node representation ; Fragment-aware readout: For each atom, its representation is concatenated with the parent fragment representation, the atom attention weights are calculated, and the atom pooling representation is obtained. in, Represents atoms Normalized attention weights Represents atoms The final expression, This represents the final representation of the parent segment corresponding to atom 'a'. This represents vector concatenation. This represents atomic-level attention pooling. Represents a set of atomic nodes; Set up S learnable fragment query vectors qs, and perform multi-query attention readout on the fragment representations: in, This represents fragment-level multi-query pooling representation. Indicates the first The pooled result obtained from the fragment query, where S represents the number of learnable fragment queries. Indicates the first A query is assigned to a fragment. Normalized attention weights Representing fragments The final expression, Indicates the first A learnable fragment query vector, This indicates that normalization is performed on all segments. Represents the set of fragment nodes; S is set to 4, and the atomic pooling representation, fragment pooling representation, and virtual global node representation are concatenated, and the downstream molecular representation is obtained through a multilayer perceptron: Fine-tuning and output: Input a prediction head corresponding to the target task. For binary or multi-label classification tasks, the prediction head outputs one or more logarithmic values ​​for each category and is trained using binary cross-entropy with logarithmic values. For continuous molecular property regression tasks, the prediction head outputs one or more continuous values ​​and is trained using mean squared error. For tasks that use absolute error as the evaluation target, L1 loss is used. During inference, S1 to S5 and the above readout process are repeated for the molecules to be predicted, and the corresponding category probability or continuous property prediction value is output.

9. A molecular property prediction system based on hierarchical segment-guided cross-attention, used to implement the molecular property prediction method based on hierarchical segment-guided cross-attention as described in any one of claims 1-8, characterized in that, include: The molecular diagram generation module is used to convert SMILES structure strings into atomic-level molecular diagrams; The fragmentation module is used to perform BRICS key breaking, small fragment merging, and backup splitting; The molecular-fragment graph construction module is used to generate fragment graphs and cross-layer membership edges; The feature generation module is used to generate atomic, chemical bond, fragment, and molecular-level features. A hierarchical coding module is used to perform multiple interleaved coding layers; The pre-training module is used to jointly compute the three types of self-supervised losses and update the model parameters; The fine-tuning prediction module is used to fuse multi-granularity representations and output molecular property prediction results.

Images