Task-aware functional group interaction modeling method based on large model device

CN122886668APending Publication Date: 2026-10-09DALIAN UNIV OF TECH +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611225734.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-13
Publication Date
2026-10-09

AI Technical Summary

Technical Problem

[0006]本发明针对现有方法难以有效识别关键官能团、建模其交互关系并将其转化为分子属性预测结果的问题,提出一种基于大模型装置的任务感知官能团交互建模方法(Motif-of-Thought,MoT)

Benefits of technology

(1)针对现有方法依赖静态规则划分官能团,难以动态适配任务需求的问题,本发明设计分词器聚合局部原子环境形成官能团,并采用自适应熵门控技术量化分子中所有官能团的分布熵,以此自适应增强属性相关官能团的权重,实现官能团动态划分与任务相关性筛选,解决了传统固定划分方式与任务脱节的问题。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122886668A_ABST
    Figure CN122886668A_ABST
Patent Text Reader

Abstract

The present application belongs to the technical field of molecular representation learning and graph large model, and relates to a task-aware functional group interaction modeling method based on a large model device. The present application dynamically divides the molecular structure based on a graph neural network-based functional group tokenizer, encodes the semantic embedding representation of atoms and functional groups, uses an adaptive entropy gate to evaluate the importance of each functional group and differentially weights the functional groups, so as to enhance the attention to task-critical functional groups. Finally, the functional group representation is input into a large language model device deployed on a server side, the device captures the interaction relationship between the functional groups in real time and completes reasoning, and outputs the molecular attribute prediction result and the traceable key functional group structure, thereby improving the accuracy and interpretability of the molecular attribute prediction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of molecular representation learning and graph large model technology, and relates to a task-aware functional group interaction modeling method based on a large model device. Background Technology

[0002] Molecular representation learning (MRL) is the process of transforming information about a molecule's chemical structure, properties, and interactions into a computable vector representation. In chemical tasks such as drug design, molecular property prediction, reaction prediction, and materials discovery, MRL provides tractable structured inputs for models, serving as a crucial foundation for efficient computation and prediction. To better accomplish these tasks, it is necessary to effectively encode the chemical information contained in molecules, such as atomic topology, chemical bond types, and functional groups, into information-rich vector representations for use by downstream models. The emergence of Graph Large Language Models (GLLMs) offers a new approach to capturing implicit structural dependencies within molecules to optimize their vector representations. GLLMs, with their powerful inference capabilities gained through pre-training and their ability to transfer prior knowledge, demonstrate significant potential in various downstream application scenarios.

[0003] In molecular property prediction tasks, the functional groups formed by atoms and their surrounding chemical reaction environment have a significant impact on the prediction results. Existing methods typically classify functional groups based on limited chemical rules (such as Recap, BRICS, etc.), statically segmenting a general set of functional groups by breaking specific chemical bonds, and encoding all functional groups into the molecular representation input model. However, the functional groups extracted by such methods often contain redundancy or repetition, while the properties of a molecule are fundamentally determined by the interactions between a few key functional groups. For example, the good water solubility of ethylene glycol (HOCH2CH2OH) is mainly due to the hydrophilic structure formed by the intramolecular hydrogen bonding between two hydroxyl groups (-OH), while nonpolar functional groups such as methylene (-CH2-) contribute very little to water solubility. Therefore, dynamically identifying task-relevant key functional groups and their interactions is of great significance for improving the accuracy and efficiency of molecular property prediction.

[0004] Currently, molecular representation learning methods based on GLLMs mainly focus on constructing pre-trained instruction sets to align molecular representations with natural language. While these methods enhance the model's understanding of molecular semantics, they fail to fully utilize the self-attention mechanism of large language models to capture key interactions of functional groups within molecules. For example, pre-training methods based on the bioactivity database (ChEMBL) (GIMET) and pre-training methods based on cross-modal biomolecular instruction sets (Mol-instructions) both train large language models by designing specialized chemical text instructions to enhance their semantic association with molecular embeddings. Multi-level chemical knowledge large models (KnowMol) further utilize atomic, functional group, and overall molecular information for molecular-text alignment, thereby optimizing the quality of pre-training data. However, these methods, while focusing on achieving molecular-text representation alignment, fail to reveal how large language models utilize prior knowledge to model functional group interactions at the molecular structure level and complete interpretable reasoning processes.

[0005] Recently, some studies have attempted to introduce functional group information into large language models to guide them in completing downstream tasks, such as the pre-training method to alleviate functional group illusion (HIGHT) and the molecular hierarchical structure understanding method based on custom functional groups (LLaMo). However, these methods still rely on static rules to classify functional groups and input all functional groups indiscriminately into large language models, making it difficult for the models to identify key functional groups and establish the correlation between interactions between functional groups and target chemical properties. Therefore, using large models to capture functional group interactions and achieve chemical property-based reasoning still faces two major challenges: first, existing functional group classification methods are fixed and task-independent, requiring dynamic adjustments based on the actual chemical environment within the molecule and the needs of downstream tasks; second, current reasoning processes fail to effectively capture interactions between functional groups at the molecular structure level and struggle to link these interactions with specific chemical properties, resulting in insufficient accuracy of the models when inferring molecular properties. Summary of the Invention

[0006] This invention addresses the challenge of existing methods in effectively identifying key functional groups, modeling their interactions, and transforming them into molecular property predictions. It proposes a task-aware functional group interaction modeling method (Motif-of-Thought, MoT) based on a large-scale model device. This method learns key functional groups and their representations related to molecular properties through a graph neural network functional group segmenter and an adaptive entropy gating mechanism. Building upon this, a large-scale language model device deployed on a server captures the interactions between functional groups. While outputting prediction results, it provides functional group-level reasoning based on chemical knowledge, significantly improving the interpretability and reliability of the model predictions.

[0007] The basic process is as follows: A graph neural network-based motif tokenizer dynamically segments the molecular structure and encodes semantic embeddings of atoms and functional groups. Adaptive Entropy Gating (AEG) is used to evaluate the importance of each segmented functional group and apply differentiated weighting to enhance focus on key functional groups. Finally, the functional group representations are input into a large language model device deployed on a server. This device captures the interactions between functional groups in real time and performs inference, outputting molecular attribute prediction results and traceable key functional group structures, thereby improving the accuracy and interpretability of molecular attribute prediction.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for modeling the interaction of task-aware functional groups based on a large model device includes the following steps: Step 100: The simplified linear molecular representations (smiles) of molecules in the dataset are transformed into molecular graph structures, serving as the basic input for the subsequent functional group word segmentation. Based on the background knowledge of molecular attributes corresponding to different datasets, a molecular attribute question-answering dataset is constructed as the instruction set for subsequent training and validation of the overall model. Specifically: Step 101, Construct the molecular graph. Construct the corresponding molecular graph based on the molecule's SMILE expression, denoted as [equation missing]. ,in A molecular diagram representing the target molecule. It is contained in the molecule A collection of atoms, It is the set of edges in the molecule. This represents the initial node feature matrix, used to describe the attribute information of the nodes in the molecular graph, where... Represents the dimension of node features.

[0009] Step 102: Construct a molecular attribute question-and-answer dataset. Based on the molecular graph from Step 101, construct question-and-answer instructions for training and validating the overall model. : Among them, the background knowledge related to molecular properties is denoted as The question that queries the large language model is denoted as The correct answer to the question is denoted as Other text content is uniformly recorded as When training the model using a question-answering dataset, freeze the backbone parameters of the large language model and retain only its reasoning ability.

[0010] Step 200: Input the molecular graph set constructed in Step 100 into the functional group word segmentation model to obtain the dynamic functional group vocabulary segmentation matrix for each molecule and the vector representation of each functional group. Specifically: Step 201, take the molecular map obtained in step 100. The input graph is processed by a functional group-based neural network tokenizer, which encodes atomic-level representations. To explicitly introduce constraints on atomic valence states (i.e., degrees) during information transmission, the encoder employs a degree-dependent aggregated weight matrix. The matrix and atoms degree Related atoms with the same degree of polymerization share the same aggregation weight for feature updates, thus obtaining a representation that accurately reflects the local chemical environment and bonding state of the molecule. Based on this, the target molecule map... Middle Atom The representation of In the encoder The layer update process is defined as follows: in, Indicates the index of the encoder layer. This represents the total number of layers in the encoder. Represents atoms The set of neighboring atoms. During the aggregation process, Representing neighboring atoms In the Features of the layer Represents atoms With atoms Edge characteristics and symbols between This represents a vector concatenation operation. This represents a non-linear activation function.

[0011] The atomic-level representation of the output of the last functional group word segmenter is as follows: ,in This represents the mapping function of the functional group segmenter. This output provides the basis for subsequent functional group segmentation and representation.

[0012] Step 202: Calculate the functional group vocabulary segmentation matrix. Based on the atomic-level representation output in step 201, the model learns the functional group mapping matrix during training according to downstream tasks. This matrix is ​​used to... The representation of an atom is mapped to The latent space of each functional group is calculated, and the partition score of each atom with respect to each functional group is calculated through a softmax operation. These scores form the partition vector of that atom. The partitioning vector represents the importance of an atom to each candidate functional group; a higher score indicates a greater likelihood that the atom belongs to that functional group. In the functional group segmenter... The division of each atom into layers is calculated using the following method: in, Indicates the functional groups after division. The functional group mapping matrix, This is a bias term. The molecule's... One atom The partitioning fraction matrix that together constitutes the whole .

[0013] Step 203, based on the atomic representation obtained in step 201 and the functional group mapping matrix obtained in step 202 Vector representation of each functional group Defined as: Finally, the functional group eigenvalue representation matrix is: The vector representation of each functional group is as follows: .

[0014] Step 300: Based on the functional group vocabulary segmentation matrix obtained in Step 200, calculate the task relevance score for each functional group. Further, based on the differences in the distribution probability of functional groups in the molecule, calculate entropy gating to provide a foundation for subsequent adaptive weighted key functional group representation. Specifically: Step 301: Calculate the task relevance score for each functional group. In the functional group word segmenter... Layer, based on the overall partition score matrix obtained in step 200 The task relevance score vector for that layer is obtained by summing the partition scores of each functional group along the atomic dimension. The scores from each layer are summed to form the final score vector. , of which scalar of a functional group Quantified the first Functional group The degree of correlation between the functional group and the target attribute. The higher the task relevance score, the more important the functional group is in predicting the target attribute.

[0015] Step 302: Based on the task relevance score of each functional group in step 301, calculate... The probability distribution of functional groups in the molecule Building upon this, normalized entropy is introduced to quantify the uncertainty in the distribution of functional groups within molecules, representing the concentration of key motifs. Normalized entropy Defined as: in, express The first of the functional groups The probability distribution of each functional group. This is a constant used to ensure numerical stability. Based on the normalized entropy, the adaptive gating factor, i.e., entropy gating, is calculated. ,in This factor is used to calibrate the representation of functional groups. Its mechanism of action is to enhance the contribution of key functional group features when the functional group distribution is concentrated (low entropy), and to fall back to the original context features through the gating factor when the functional group distribution is ambiguous (high entropy).

[0016] Step 400: In the adaptive entropy module, based on the entropy gating and functional group distribution probability obtained in step 300 and the functional group representation obtained in step 200, the final functional group representation is obtained by aligning the functional group representation with the natural language representation space. Specifically: Step 401: Obtain the final functional group representation. This is based on the entropy gating calculated in step 300. Adaptive use of functional group distribution probability The functional group representation output by weighted step 200 is used to obtain the corrected functional group representation. , Defined as: in, This represents the dot product. The modified functional group representation is then mapped to the natural language space through a linear layer to obtain the final functional group representation.

[0017] Step 500: The functional group representations obtained in step 400 are input into the large language model device, which then performs functional group interaction capture, inference calculation, and result output. Specifically: Step 501: Model the interaction strength between key functional groups. Based on the functional group representation obtained in Step 400, the interaction strength of any pair of functional groups is modeled by averaging the attention scores of all layers and all attention heads in the large language model. Functional Groups and sensual groups Interaction strength between Defined as: The higher the interaction strength, the greater its importance to the target attribute prediction task. This strength is calculated in real time by the server-side large language model and can be used to backtrack the interaction relationships of key functional groups.

[0018] Step 502: The large language model outputs the prediction results and interaction information. This involves combining the text set from the attribute question-and-answer instruction set constructed in step 100. and the functional groups obtained in step 400 These are input into the large language model device. The large language model performs inference through likelihood estimation, generates molecular attribute prediction results, and simultaneously outputs the attention weights of each layer to reconstruct the functional group interaction process. This process is defined as follows: in, Representing a large language model, The first one generated by the large language model One tag, This indicates the probability of generating the tag. Indicates the first The sequence of tags generated before each tag, It is the length of the sequence generated by the large language model.

[0019] The beneficial effects of this invention are as follows: (1) In view of the problem that existing methods rely on static rules to divide functional groups and are difficult to dynamically adapt to task requirements, this invention designs a word segmenter to aggregate local atomic environments to form functional groups, and adopts adaptive entropy gating technology to quantify the distribution entropy of all functional groups in the molecule, thereby adaptively enhancing the weight of attribute-related functional groups, realizing dynamic division of functional groups and task-related screening, and solving the problem of the disconnect between traditional fixed division methods and tasks.

[0020] (2) In response to the challenge that the interaction between functional groups is difficult to capture and is not sufficiently related to chemical properties, the present invention inputs the weighted functional group representation into a large language model device, and captures key functional group interactions by stimulating the attention mechanism through chemical-related knowledge in the instruction set, so that the model can deduce the reasoning basis related to chemical properties from the structural level, thereby improving the accuracy and interpretability of molecular property prediction.

[0021] (3) The innovations of this invention are: 1) By using a word segmenter to aggregate local atomic representations into functional group representations, dynamic division of functional groups in molecules is achieved, breaking through the limitations of traditional static rule-based functional group division; 2) Based on information theory, an adaptive entropy gating technique is designed to suppress structural noise and enhance attribute-related signals by quantifying the functional group distribution entropy, thus solving the problem of key functional group representation screening and weight calibration; 3) Based on weighted functional group representations, a large language model device is used to backtrack attention scores in real time and visualize the key functional group interaction structure. The resulting interaction path is highly consistent with the existing chemical knowledge system, realizing the interpretable presentation of the molecular attribute reasoning basis. Attached Figure Description

[0022] Figure 1This is a framework diagram of the method of the present invention. Detailed Implementation

[0023] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0024] The basic process of the task-aware functional group interaction modeling method based on a large model device of the present invention is as follows: Figure 1 As shown. This embodiment specifically includes the following steps: Step 100: The simplified linear molecular representations (smiles) of molecules in the dataset are transformed into molecular graph structures, which serve as the basic input for the subsequent functional group word segmentation. Based on the background knowledge of molecular attributes corresponding to different datasets, a molecular attribute question-answering dataset is constructed as the instruction set for subsequent training and validation of the overall model.

[0025] Step 101: Construct the molecular graph. Based on the molecular property prediction benchmark dataset MoleculeNet, the tasks it covers are divided into the following categories: toxicity prediction tasks (including SIDER, ClinTox, Tox21), bioactivity prediction tasks (including BACE, HIV, MUV), metabolism prediction tasks (including BBBP, CYP450), physicochemical numerical prediction tasks (including ESOL, FreeSolv, Lipo), and quantum chemical energy prediction tasks (QM9). For each of the above tasks, the SMILES expressions of molecules in the dataset are extracted, and the corresponding molecular graph representations are constructed, denoted as . .in, A molecular diagram representing the target molecule. It is contained in the molecule A collection of atoms, It is the set of edges in the molecule. This represents the initial node feature matrix, used to describe the attribute information of the nodes in the molecular graph, where... This represents the node feature dimension. The node feature dimension is uniformly set to 150.

[0026] Step 102: Construct a molecular attribute question-and-answer dataset. Based on the molecular graph from Step 101, construct question-and-answer instructions for training and validating the overall model. : Among them, the background knowledge related to molecular properties is denoted as The question that queries the large language model is denoted as The correct answer to the question is denoted as Other text content is uniformly recorded as When training the model using this question-and-answer dataset, the backbone parameters of the large language model are frozen, retaining only its reasoning ability.

[0027] Step 200: Input the molecular graph set constructed in Step 100 into the functional group segmentation model to obtain the dynamic functional group partitioning matrix and vector representation of each molecule. Specifically: Step 201, take the molecular map obtained in step 100. The input graph is processed by a functional group-based neural network tokenizer, which encodes atomic-level representations. To explicitly introduce constraints on atomic valence states (i.e., degrees) during information transmission, the encoder employs a degree-dependent aggregated weight matrix. The matrix and atoms degree The system is associated with a degree range of 1 to 5, where atoms with the same degree share the same polymerization weight for feature updates, in order to obtain a representation that accurately reflects the local chemical environment and bonding state of the molecule.

[0028] Based on this, the target molecule map Middle Atom The representation of In the encoder The layer update process is defined as follows: in, Indicates the index of the encoder layer. This represents the total number of layers in the encoder. The value is set to 3. Represents atoms The set of neighboring atoms. During the aggregation process, Representing neighboring atoms In the Features of the layer Represents atoms With atoms Edge characteristics and symbols between This represents a vector concatenation operation. This represents a non-linear activation function.

[0029] The atomic-level representation of the output of the last functional group word segmenter is as follows: ,in This represents the mapping function of the tokenizer. This output provides the basis for subsequent functional group partitioning and representation.

[0030] Step 202: Calculate the functional group partitioning matrix. Based on the atomic-level representation output in step 201, the model learns the functional group mapping matrix during training according to the downstream tasks. Number of functional groups Set to 32, this matrix is ​​used to map the representation of each atom to the representation space of 32 functional groups, and calculates the partition score of each atom relative to each functional group through a softmax operation. These scores form the partition vector of that atom. The partitioning vector represents the importance of an atom to each candidate functional group; a higher score indicates a greater likelihood that the atom belongs to that functional group. In the functional group segmenter... The division of each atom into layers is calculated using the following method: in, Indicates the functional groups after division. The functional group mapping matrix, This is a bias term. The molecule's... One atom The partitioning fraction matrix that together constitutes the whole .

[0031] Step 203, based on the atomic representation obtained in step 201 and the functional group mapping matrix obtained in step 202 Vector representation of each functional group Defined as: Finally, the functional group eigenvalue representation matrix is: The vector representation of each functional group is as follows: .

[0032] Step 300: Based on the functional group vocabulary segmentation matrix obtained in Step 200, calculate the task relevance score for each functional group. Further, based on the differences in the distribution probability of functional groups in the molecule, calculate entropy gating to provide a foundation for subsequent adaptive weighted key functional group representation. Specifically: Step 301: Calculate the task relevance score for each functional group. In the functional group word segmenter... Layer, based on the overall partition score matrix obtained in step 200 The task relevance score vector for that layer is obtained by summing the partition scores of each functional group along the atomic dimension. The scores from each layer are summed to form the final score vector. Each scalar Quantified functional groups The degree of correlation between the functional group and the target attribute. The higher the task relevance score, the more important the functional group is in predicting the target attribute.

[0033] Step 302: Based on the task relevance score of each functional group in step 301, calculate... The probability distribution of functional groups in the molecule Building upon this, normalized entropy is introduced to quantify the uncertainty in the distribution of functional groups within molecules, representing the concentration of key motifs. Normalized entropy Defined as: in, express The first of the functional groups The probability distribution of each functional group. This is a constant used to ensure numerical stability. The adaptive gating factor is calculated based on the normalized entropy. ,in This factor is used to calibrate the representation of functional groups. Its mechanism of action is to enhance the contribution of key functional group features when the functional group distribution is concentrated (low entropy), and to fall back to the original context features through the gating factor when the functional group distribution is ambiguous (high entropy).

[0034] Step 400: In the adaptive entropy module, based on the entropy gating and functional group distribution probability obtained in step 300 and the functional group representation obtained in step 200, the final functional group representation is obtained by aligning the functional group representation with the natural language representation space. Specifically: Step 401: Obtain the final functional group representation. This is based on the entropy gating calculated in step 300. Adaptive use of functional group distribution probability The functional group representation output by weighted step 200 is used to obtain the corrected functional group representation. , Defined as: in, This represents the dot product. The modified functional group representation is then mapped to the natural language space through a linear layer to obtain the final functional group representation.

[0035] Step 500: The functional group representations obtained in Step 400 are input into a large language model device deployed on a server equipped with an NVIDIA A100 (80G) graphics processing unit. This device is responsible for capturing functional group interactions, performing inference calculations, and outputting the results. Specifically: Step 501: Model the interaction strength between key functional groups. Based on the functional group representation obtained in Step 400, the interaction strength of any pair of functional groups is modeled by averaging the attention scores of all layers and all attention heads in the large language model. Functional Groups and sensual groups Interaction strength between Defined as: The higher the interaction strength, the greater its importance to the target attribute prediction task. This strength is calculated in real time by the large language model and can be used to backtrack the interaction relationships of key functional groups.

[0036] Step 502: The large language model outputs the prediction results and interaction information. This involves combining the text set from the attribute question-and-answer instruction set constructed in step 100. and the functional groups obtained in step 400 These are input into the large language model device. The large language model performs inference through likelihood estimation, generates molecular attribute prediction results, and simultaneously outputs the attention weights of each layer to reconstruct the functional group interaction process. This process is defined as follows: in, Representing a large language model, The first one generated by the large language model One tag, This indicates the probability of generating the tag. Indicates the first The tag sequence generated before each tag, It is the length of the sequence generated by the large language model.

[0037] Verification results: In the experiments of this invention, comparisons were made with 11 state-of-the-art methods on 12 datasets. The performance of each method was evaluated based on three metrics: area under the ROC curve (AUC), MAE, and RMSE, in order to verify the effectiveness of this invention.

[0038] The specific information about the dataset is shown in Table 1.

[0039] Table 1 Dataset

[0040] Dataset introduction: (1) SIDER dataset: SIDER data contains drug side effect data, including information on the correlation between marketed drugs and adverse reactions. This data is primarily used to predict potential drug side effects and is crucial for assessing drug safety and providing risk warnings.

[0041] (2) ClinTox dataset: ClinTox data includes FDA-approved drugs as well as those that failed in clinical trials due to toxicity issues. This data is primarily used to differentiate between drugs with varying clinical toxicity.

[0042] (3) Tox21 dataset: The Tox21 data contains data on the toxicity interference of compounds on 12 biological pathways, which is used to test compound toxicity prediction models.

[0043] (4) BACE dataset: BACE provides molecular data on inhibitory activity against β-secretase 1 (BACE1, a key target for Alzheimer's disease treatment). This dataset is used for enzyme inhibitor prediction to rapidly discover and optimize potential BACE1 inhibitors.

[0044] (5) HIV dataset: HIV includes bioactivity data of HIV inhibitors, used to evaluate the model's performance in predicting antiviral bioactivity.

[0045] (6) MUV dataset: The MUV dataset contains 17 challenging tasks, each representing a biological activity or chemical property. These compounds exhibit varying performance across tasks, providing a rich sample of test cases. This dataset is used to evaluate the effectiveness and accuracy of different virtual screening methods.

[0046] (7) BBBP dataset: The BBBP dataset uses binary labels to indicate whether compounds can cross the blood-brain barrier. This dataset is important for developing drugs targeting the central nervous system or avoiding central nervous system side effects.

[0047] (8) CYP450 dataset: CYP450 data contains information on whether a compound is metabolized by the cytochrome P450 enzyme system. This data is used to assess drug metabolism rates, potential interactions, and safety.

[0048] (9) ESOL dataset: The ESOL dataset contains experimental water solubility data for small molecule compounds. This dataset is used to test the accuracy of models predicting solubility directly from molecular structure.

[0049] (10) FreeSolv dataset: FreeSolv data provides experimental dissolution free energies of small molecules in water. It is used to calibrate and test computational chemistry methods and physics-driven solubility prediction models.

[0050] (11) Lipo dataset: Lipo data contains measurements of the lipid-water partition coefficient of compounds. This data is used to describe the lipophilicity of molecules and to predict the absorption and distribution behavior of drugs in vivo.

[0051] (12) QM9 dataset: The QM9 dataset contains quantum mechanical properties of molecules in computational chemistry, such as molecular orbital energy levels and dipole moments. This data is used to study the electronic structure of small molecules.

[0052] The proposed method was systematically evaluated on 12 MoleculeNet datasets across 5 tasks. The results are summarized in Tables 2 and 3. Table 2 shows the performance comparison results of the AUC-ROC index in the classification task, and Table 3 shows the performance comparison results of the MAE and RMSE in the regression task. In the above evaluation, this embodiment selected 5 molecular graph expert models (KV-PLM, MoMu, Uni-Mol, MolFM, GIMLET), 3 basic large language models (Vicuna-v1.3-7B, LLaMA-2-7B-chat, Vicuna-13B), and 2 representative molecular graph large language models (Mol-Instruction, InstructMol, HIGHT) as baseline models for comprehensive comparison with the proposed MoT method. The best results in each table are marked in bold. The comparison method is described below: (1) KV-PLM: By explicitly injecting molecular structure priors during the pre-training stage of the molecular language model, atomic information can directly participate in representation learning in attention modeling, thus achieving early deep integration of structure and semantics in molecular representation.

[0053] (2) MoMu: It adopts a dual encoder architecture of molecular encoder and text encoder, and aligns molecular structure representation and text semantics in a shared embedding space through contrastive learning to achieve cross-modal fusion of molecular representation.

[0054] (3) Uni-Mol: By encoding the three-dimensional conformation of molecules, more accurate chemical spatial structure information can be obtained, thereby improving the performance of downstream property prediction tasks.

[0055] (4) MolFM: By introducing explicit feature interaction and attention fusion mechanism in the middle layer of molecular representation, information flow between different structural components is enhanced, thereby refining molecular structural feature fusion in a task-driven manner.

[0056] (5) GIMLET: In the attention mechanism, positional encoding is used to separate the graph structure and instruction text encoding, thereby realizing a model with generalization ability on molecular zero-shot learning and general tasks.

[0057] (6) Vicuna: An open-source large-scale language model, which varies in terms of the number of parameters and inference capabilities depending on its size. For example, version 13B usually performs better in complex language understanding and generation tasks due to its larger parameter size, while version 7B has a faster inference speed.

[0058] (7) LLaMA-2-7B-chat: A conversational model based on the LLaMA-2 architecture for fine-tuning instructions, which is an important benchmark for evaluating the molecular reasoning ability of general large language models.

[0059] (8) Mol-Instruction: Mol-Instruction constructs a large-scale molecular instruction dataset and trains it in conjunction with a large language model.

[0060] (9) InstructMol: InstructMol enhances the model’s understanding of molecular structure semantics and task instructions by integrating molecular graph structures and sequences as vector inputs into the basic large language model.

[0061] (10) HIGHT: By using a hierarchical graph tokenization strategy, molecular information of graph structures is integrated into the model to capture the features of molecular hierarchical structures, thereby improving the structural perception ability of large language models on molecular tasks.

[0062] The experimental results in Tables 2 and 3 show that the MoT method outperforms the three baseline models across all 12 datasets. Compared to basic large language models (such as Vicuna-v1.3-7B, LLaMA-2-7B-chat, and Vicuna-13B) and molecular graph large language models (such as Mol-Instruction, InstructMol, and HIGHT), MoT achieves the best results on all datasets and task evaluation metrics, indicating that this method can not only identify key functional groups in classification tasks but also effectively model the cumulative effects of local structural interactions in regression tasks, thus significantly improving overall reasoning ability. Furthermore, compared to molecular graph expert models (such as KV-PLM, MoMu, Uni-Mol, MolFM, and GIMLET), MoT performs better in terms of MAE and RMSE metrics in regression tasks, demonstrating that even without large-scale pre-training, introducing a large language model can still capture the interaction information of key structures at the chemical mechanism level, improving task performance while reducing computational resource requirements. MoT's advantages mainly stem from the following two improvements: (1) Existing baseline methods generally rely on large-scale pre-training corpora, and the knowledge they learn is often general rather than the key chemical knowledge necessary for downstream tasks. MoT achieves dynamic segmentation of functional groups and task-related screening through functional group segmentation and adaptive entropy gating technology, overcoming the problem of the traditional fixed segmentation method being disconnected from the task objective. (2) In the inference stage, by utilizing the self-attention mechanism of large language models, it can naturally model and capture the interactions between key functional groups, a process that is consistent with the basic principle of the interaction influence of functional groups in chemistry. Although some individual indicators are slightly lower in some baseline comparisons, MoT requires significantly less computational resources and training time, and has the best overall performance, especially in regression tasks such as ESOL, which always maintains its advantage, verifying that this method has good generalization ability and robustness in molecular property prediction tasks.

[0063] Table 2. Comparison of AUC-ROC metrics for different methods on classification task datasets. (The higher the indicator value, the better; the optimal result is marked in bold.)

[0064] Table 3. Comparison of MAE and RMSE metrics for different methods on regression task datasets. (The smaller the indicator value, the better; the optimal result is marked in bold.)

Claims

1. A method for modeling the interaction of task-aware functional groups based on a large-scale model device, characterized in that, Includes the following steps: Step 100: The simplified linear molecular representations (smiles) of molecules in the dataset are transformed into molecular graph structures, which serve as the basic input for the subsequent functional group word segmenter. Based on the background knowledge of molecular attributes corresponding to different datasets, a molecular attribute question-answering dataset is constructed as the instruction set for subsequent training and validation of the overall model. Step 200: Input the molecular graph set constructed in step 100 into the functional group word segmentation model to obtain the dynamic functional group word segmentation matrix of each molecule and the vector representation of each functional group. Step 300: Based on the functional group vocabulary segmentation matrix obtained in step 200, calculate the task relevance score for each functional group; further, based on the differences in the distribution probability of functional groups in the molecule, calculate entropy gating to provide a basis for subsequent adaptive weighted key functional group representation. Step 400: In the adaptive entropy module, based on the entropy gating, functional group distribution probability obtained in step 300 and the functional group representation obtained in step 200, the final functional group representation is obtained by aligning the functional group representation with the natural language representation space. Step 500: Input the functional group representation obtained in step 400 into the large language model device, which then completes the functional group interaction capture, inference calculation and result output.

2. The task-aware functional group interaction modeling method based on a large model device according to claim 1, characterized in that, Step 100 is as follows: Step 101, Construct the molecular graph; construct the corresponding molecular graph based on the molecule's SMILE expression, denoted as ,in A molecular diagram representing the target molecule. It is contained in the molecule A collection of atoms, It is the set of edges in the molecule; This represents the initial node feature matrix, used to describe the attribute information of the nodes in the molecular graph, where... Indicates the dimension of node features; Step 102: Construct a molecular attribute question-and-answer dataset; based on the molecular graph from Step 101, construct question-and-answer instructions for training and validating the overall model. : Among them, the background knowledge related to molecular properties is denoted as The question that queries the large language model is denoted as The correct answer to the question is denoted as All other text content is uniformly recorded as When training a model using a question-and-answer dataset, freeze the backbone parameters of the large language model and retain only its reasoning ability.

3. The task-aware functional group interaction modeling method based on a large model device according to claim 1, characterized in that, Step 200 is as follows: Step 201, take the molecular map obtained in step 100. The input graph is processed by a neural network functional group word segmenter, which encodes atomic-level representations. To explicitly introduce atomic valence state constraints in information transmission, the encoder employs a degree-dependent aggregated weight matrix. The matrix and atoms degree Related atoms with the same degree of polymerization share the same aggregation weight for feature updates, in order to obtain a representation that accurately reflects the local chemical environment and bonding state of the molecule; target molecule map Middle Atom The representation of In the encoder The layer update process is defined as follows: in, Indicates the index of the encoder layer. This represents the total number of layers in the encoder. Represents atoms The set of neighboring atoms; during the aggregation process, Representing neighboring atoms In the Features of the layer Represents atoms With atoms Edge characteristics and symbols between This represents a vector concatenation operation. Represents a non-linear activation function; The atomic-level representation of the output of the last functional group word segmenter is as follows: ,in This represents the mapping function of the functional group word segmenter; its output provides the basis for subsequent functional group segmentation and representation. Step 202: Calculate the functional group vocabulary segmentation matrix; based on the atomic-level representation output in step 201, the model will learn the functional group mapping matrix during training according to the downstream tasks. This matrix is ​​used to... The representation of an atom is mapped to The latent space of each functional group is calculated, and the partition score of each atom with respect to each functional group is calculated through a softmax operation. These scores form the partition vector of that atom. ; In the functional group word segmenter The division of each atom into layers is calculated using the following method: in, Indicates the functional groups after division. The functional group mapping matrix, For bias terms; the molecule's One atom The partitioning fraction matrix that together constitutes the whole ; Step 203, based on the atomic representation obtained in step 201 and the functional group mapping matrix obtained in step 202 Vector representation of each functional group Defined as: Finally, the functional group feature representation matrix is: The vector representation of each functional group is as follows: .

4. The task-aware functional group interaction modeling method based on a large model device according to claim 1, characterized in that, Step 300 is as follows: Step 301: Calculate the task relevance score for each functional group; in the functional group word segmenter... Layer, based on the overall partition score matrix obtained in step 200 The task relevance score vector for that layer is obtained by summing the partition scores of each functional group along the atomic dimension. The scores from each level are summed to form the final score vector. , of which scalar of a functional group Quantified the first Functional group The degree of correlation between the target attribute and the target attribute; Step 302: Based on the task relevance score of each functional group in step 301, calculate... The probability distribution of functional groups in the molecule Normalized entropy is introduced to quantify the uncertainty of functional group distribution in molecules, representing the concentration of key motifs; Normalized entropy Defined as: in, express The first of the functional groups The probability distribution of each functional group; It is a constant used to ensure numerical stability; based on normalized entropy, the adaptive gating factor is calculated, i.e., entropy gating. ,in .

5. The task-aware functional group interaction modeling method based on a large model device according to claim 1, characterized in that, Step 400 is as follows: Step 401: Obtain the final functional group representation; based on the entropy gating calculated in step 300. Adaptive use of functional group distribution probability The functional group representation output by weighted step 200 is used to obtain the corrected functional group representation. , Defined as: in, The dot product is represented; the modified functional group representation is mapped to the natural language space through a linear layer to obtain the final functional group representation.

6. The task-aware functional group interaction modeling method based on a large model device according to claim 1, characterized in that, Step 500 is as follows: Step 501: Model the interaction strength between key functional groups; based on the functional group representation obtained in step 400, for any pair of functional groups, the interaction strength is modeled by averaging the attention scores of all layers and all attention heads in the large language model; functional groups and sensual groups Interaction strength between Defined as: Step 502: The large language model outputs the prediction results and interaction information; the text set in the attribute question-answering instruction set constructed in step 100 is then processed. and the functional groups obtained in step 400 Input them together into the large language model device; The large language model performs inference through likelihood estimation, generates molecular attribute prediction results, and outputs attention weights for each layer to reconstruct the functional group interaction process; this process is defined as: in, Representing a large language model, The first one generated by the large language model One tag, This indicates the probability of generating the tag. Indicates the first The sequence of tags generated before each tag, It is the length of the sequence generated by the large language model.