Processing method and apparatus for stacked ensemble model for molecular property prediction

Abstract

Embodiments of the present application relate to a processing method and device for a stacked integrated model for molecular attribute prediction, the method comprising: constructing an integrated model based on a two-stage stacking mechanism, denoted as a corresponding first integrated model; performing model training on the first integrated model based on a preset original data set; and if the model training is successful, performing molecular attribute prediction processing based on the first integrated model. The present application can solve the problems of single model structure and single molecular representation scale in conventional solutions.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a processing method and apparatus for a stacked ensemble model for molecular property prediction. Background Technology

[0002] Quantitative structure-activity relationship (QSAR) is an important component of drug discovery, environmental toxicology, and chemical risk assessment. In QSAR analysis, a set of compounds with known molecular properties is used to build a statistical model that correlates the molecular structure of a compound with its properties. This model is then used to predict the corresponding molecular properties based on the molecular structure. In practice, we have found that such models often have the following drawbacks: 1) The model structure is limited; these models are either implemented based on a simple machine learning model or a single neural network model; 2) The molecular representation scale of the model input is limited; it is either a one-dimensional molecular sequence in a format similar to SMILES, a molecular descriptor and / or molecular fingerprint with one-dimensional feature data, or a 2D molecular diagram with two-dimensional feature data, and it lacks support for 3D molecular conformations. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of existing technologies by providing a method, apparatus, electronic device, and computer-readable storage medium for processing a stacked ensemble model for molecular property prediction. Based on ensemble model theory, a two-level stacked ensemble model framework is constructed. In the first level, multiple parallel base learners (machine learning models, neural network models, or pre-trained models) are introduced to predict molecular properties from multi-scale molecular representations (molecular fingerprints, molecular descriptors, 1D molecular sequences, 2D molecular diagrams, and 3D molecular conformations), and the prediction results of each base learner in the first level are scored. In the second level, multiple parallel meta-learners (machine learning models) are introduced to further refine the coarse prediction results from the first level, and the final molecular property prediction result is obtained by averaging all refined prediction results. Before applying this ensemble model, it is trained using an ensemble training method. This invention solves the problems of single model structure and single molecular representation scale in conventional schemes. Furthermore, the ensemble model prediction method of stacking models at each level and progressively advancing the two-level models further improves the model's generalization ability and prediction accuracy.

[0004] To achieve the above objectives, a first aspect of the present invention provides a method for processing stacked ensemble models for molecular property prediction, the method comprising:

[0005] The ensemble model constructed based on the two-level stacking mechanism is denoted as the corresponding first ensemble model.

[0006] The first ensemble model is trained based on a pre-set raw dataset;

[0007] If the model is successfully trained, molecular property prediction processing is performed based on the first ensemble model.

[0008] Preferably, the first integrated model includes a data preprocessing module, a molecular multimodal characterization processing module, a first-level stacked base learner processing module, and a second-level stacked meta-learner processing module; the molecular multimodal characterization processing module includes a molecular fingerprint processing unit, a molecular descriptor processing unit, a one-dimensional molecular sequence processing unit, a two-dimensional molecular graph processing unit, and a three-dimensional molecular conformation processing unit; the first-level stacked base learner processing module includes a multimodal fusion unit, multiple parallel stacked base learners, and an evaluation unit; the second-level stacked meta-learner processing module includes a first-level prediction screening unit, multiple parallel stacked meta-learners, and a second-level prediction averaging unit;

[0009] The input terminals of the molecular fingerprint processing unit, the molecular descriptor processing unit, the one-dimensional molecular sequence processing unit, the two-dimensional molecular graph processing unit, and the three-dimensional molecular conformation processing unit are respectively connected to the data preprocessing module, and their output terminals are respectively connected to the multimodal fusion unit; the multiple output terminals of the multimodal fusion unit are respectively connected to the input terminals of each of the base learners; the output terminals of each of the base learners are connected to the input terminal of the evaluation unit; the output terminal of the evaluation unit is connected to the input terminal of the first-level prediction screening unit; the multiple output terminals of the first-level prediction screening unit are connected to the input terminals of each of the meta-learners; the output terminals of each of the meta-learners are connected to the input terminal of the second-level prediction averaging unit; the output terminal of the second-level prediction averaging unit is the model output terminal of the first ensemble model.

[0010] The base learner is a first-level molecular attribute prediction model based on a machine learning model, a neural network model, or a pre-trained model, which takes a fused feature tensor that integrates multimodal molecular features as input. The machine learning model includes GBDT model, extraTree model, LR model, MLP neural network and SVM model, and the pre-trained model includes BERT model, graph neural network model, SE(3) equal variation neural network model and Uni-Mol model. The meta-learner is a second-level molecular attribute prediction model based on a machine learning model, which takes a primary screening molecular attribute tensor as input. The machine learning model includes GBDT model, extraTree model, LR model, MLP neural network and SVM model.

[0011] Preferably, the step of training the first ensemble model based on a preset original dataset specifically includes:

[0012] The original dataset is input into the data preprocessing module for molecular data anomaly detection and label data normalization to obtain a corresponding preprocessed dataset. The original dataset contains multiple first original datasets. Each first original dataset includes a first molecular data in SMILES format and a corresponding first molecular attribute label. The first molecular attribute label includes multiple molecular attribute types and their corresponding attribute values. The preprocessed dataset contains multiple first preprocessed datasets. Each first preprocessed dataset includes a second molecular data in SMILES format and a corresponding second molecular attribute label. The second molecular attribute label includes multiple molecular attribute types and their corresponding attribute values.

[0013] The preprocessed dataset is input into the molecular multimodal characterization processing module for multimodal molecular characterization recognition processing to generate a corresponding first dataset; the first dataset includes multiple first molecular characterization data; the first molecular characterization data includes molecular fingerprint feature tensor, molecular descriptor feature tensor, one-dimensional molecular feature tensor, two-dimensional molecular graph feature tensor, three-dimensional molecular conformation feature tensor and third molecular attribute label;

[0014] The first dataset is input into the first-level stacked base learner processing module to perform first-level model training to obtain the corresponding first output dataset; the first output dataset includes multiple first base learner output data; the first base learner output data includes multiple first batch output data; the first batch output data includes a first prediction-label pair set and a first cross-validation score; the first prediction-label pair set includes multiple first prediction-label pairs; the first prediction-label pair includes a first attribute prediction tensor and the fourth molecular attribute label;

[0015] The first output dataset is input into the second-level stacked meta-learner processing module for second-level model training.

[0016] Furthermore, the step of inputting the original dataset into the data preprocessing module for molecular data anomaly detection processing and label data normalization processing to obtain the corresponding preprocessed dataset specifically includes:

[0017] The original dataset is input into the data preprocessing module;

[0018] The data preprocessing module then performs anomaly identification on all the first raw data according to the 3 sigma principle, and marks the first raw data that is not identified as anomaly as normal raw data; and extracts the first molecular attribute labels of all the normal raw data to form a corresponding normal label set;

[0019] The data preprocessing module extracts all attribute values ​​of any molecular attribute type from the normal label set to form a corresponding first numerical sequence; calculates the skewness of the first numerical sequence to obtain a corresponding first skewness; when the first skewness is less than or equal to a set threshold, normalizes the first numerical sequence based on a standard Gaussian transform to obtain a corresponding normalized numerical sequence; when the first skewness is greater than the set threshold, identifies whether the minimum value of the first numerical sequence is greater than 0; if so, normalizes the first numerical sequence based on a Box-Cox transform to obtain the corresponding normalized numerical sequence; otherwise, normalizes the first numerical sequence based on a Yeo-Johnson transform to obtain the corresponding normalized numerical sequence; and replaces the corresponding attribute values ​​in the normal label set with the normalized attribute values ​​of the obtained normalized numerical sequences.

[0020] The data preprocessing module takes each of the first molecular attribute labels that have been replaced with normalized attribute values ​​in the normal label set as the corresponding second molecular attribute label, and takes the first molecular data of the normal original data corresponding to each second molecular attribute label as the corresponding second molecular data. The first preprocessed data is composed of each second molecular data and the corresponding second molecular attribute label. The preprocessed dataset is composed of all the first preprocessed data obtained.

[0021] Furthermore, the step of inputting the preprocessed dataset into the molecular multimodal characterization processing module for multimodal molecular characterization recognition processing to generate the corresponding first dataset specifically includes:

[0022] The preprocessed dataset is input into the molecular multimodal characterization processing module;

[0023] The molecular fingerprint processing unit of the molecular multimodal characterization processing module performs molecular fingerprint feature extraction processing on the second molecular data of each of the first preprocessed data according to a preset fingerprint format to generate the corresponding molecular fingerprint feature tensor; the preset fingerprint format includes Morgan fingerprint format, ECFP fingerprint format, MACCS fingerprint format, RDKit fingerprint format and Topological fingerprint format;

[0024] The molecular descriptor processing unit of the molecular multimodal characterization processing module performs multi-class descriptor feature extraction processing on the second molecular data of each of the first preprocessed data according to a preset descriptor type set to generate the corresponding molecular descriptor feature tensor; the descriptor type set includes multiple molecular descriptor types;

[0025] The one-dimensional molecular sequence processing unit of the molecular multimodal characterization processing module uses a preset one-dimensional feature extraction model to perform one-dimensional feature extraction processing on the second molecular data of each of the first preprocessed data to generate the corresponding one-dimensional molecular feature tensor; the one-dimensional feature extraction model includes the Transformer model, the BERT-like model and the K-BERT model;

[0026] The two-dimensional molecular graph processing unit of the molecular multimodal characterization processing module uses a preset two-dimensional feature extraction model to perform two-dimensional molecular graph feature extraction processing on the second molecular data of each of the first preprocessed data to generate the corresponding two-dimensional molecular graph feature tensor; the two-dimensional feature extraction model includes the GROVER model, MOLCLR model, KPGT model and HIGNN model;

[0027] The three-dimensional molecular conformation processing unit of the molecular multimodal characterization processing module uses a preset three-dimensional feature extraction model to perform three-dimensional molecular conformation feature extraction processing on the second molecular data of each of the first preprocessed data to generate the corresponding three-dimensional molecular conformation feature tensor; the three-dimensional feature extraction model includes EGNN model, SE(3) equal variation neural network and Uni-Mol model;

[0028] The molecular multimodal characterization processing module then uses the second molecular attribute label of each of the first preprocessed data as the corresponding third molecular attribute label.

[0029] The molecular multimodal characterization processing module then uses the molecular fingerprint feature tensor, the molecular descriptor feature tensor, the one-dimensional molecular feature tensor, the two-dimensional molecular graph feature tensor, the three-dimensional molecular conformation feature tensor, and the third molecular attribute label corresponding to each of the first preprocessed data to form the corresponding first molecular characterization data; and all the obtained first molecular characterization data form the corresponding first dataset.

[0030] Furthermore, the step of inputting the first dataset into the first-level stacked base learner processing module for first-level model training to obtain the corresponding first output dataset specifically includes:

[0031] The first dataset is input into the first-level stacked base learner processing module;

[0032] The multimodal fusion unit of the first-level stacked base learner processing module creates a corresponding training dataset for each base learner, denoted as the first learner dataset. Any first molecular representation data from the first dataset is taken as the corresponding current molecular representation data. The molecular fingerprint feature tensor, molecular descriptor feature tensor, one-dimensional molecular feature tensor, two-dimensional molecular graph feature tensor, and three-dimensional molecular conformation feature tensor from the current molecular representation data are extracted to form the corresponding current feature tensor sequence. The third molecular attribute label from the current molecular representation data is extracted as the corresponding fourth molecular attribute label. The system iterates through each first correspondence record in a preset first correspondence table reflecting the correspondence between fusion tensor and base learner, taking the currently traversed first correspondence record as the current record. The first fusion tensor feature range field and the first base learner identifier field of the current record are extracted as the corresponding current feature range and current base learner identifier. The base learner identifier that matches the current base learner identifier is then selected as the current base learner identifier. The learner serves as the corresponding current base learner and performs multimodal feature fusion on one or more feature tensors in the current feature tensor sequence that match the current feature range to obtain the corresponding first molecular fusion feature tensor. The first molecular fusion feature tensor and the corresponding fourth molecular attribute label constitute the corresponding first learner data, and a correspondence is established between the first learner data and the current base learner identifier. At the end of the traversal, each obtained first learner data is stored in its corresponding first learner dataset. The first correspondence table includes multiple first correspondence records. Each first correspondence record includes a first fusion tensor feature range field and a first base learner identifier field. The first fusion tensor feature range field includes fingerprint feature type, descriptor feature type, one-dimensional feature type, two-dimensional feature type, and three-dimensional feature type. The first learner dataset corresponds one-to-one with the base learner. The first learner dataset includes multiple first learner data sets. The first learner data includes the first molecular fusion feature tensor and the fourth molecular attribute label.

[0033] The first-level stacked base learner processing module divides each of the first learner datasets into two datasets according to a preset ratio: a first training dataset and a first validation dataset. The first training dataset is then divided into multiple data subsets, denoted as corresponding single-batch training data subsets. Based on each single-batch training data subset, the corresponding base learner is trained batch by batch. In each batch of model training, Focal Loss+GHM is used. The loss function is used as the model training loss function; and at the end of each batch of model training, the base learner is used to perform molecular attribute prediction processing on each of the first molecular fusion feature tensors of the first validation dataset to generate the corresponding first attribute prediction tensor, and each of the first attribute prediction tensors and the corresponding fourth molecular attribute label forms the corresponding first prediction-label pair, and all the obtained first prediction-label pairs form the corresponding first prediction-label pair set, and the evaluation unit performs cross-validation scoring on the first prediction-label pair set to obtain the corresponding first cross-validation score, and the first prediction-label pair set and the first cross-validation score form the corresponding first batch output data; and at the end of all batches of model training, all the obtained first batch output data form the corresponding first base learner output data; and the first base learner output data corresponding to all the base learners form the corresponding first output dataset.

[0034] Furthermore, the step of inputting the first output dataset into the second-level stacked meta-learner processing module for second-level model training specifically includes:

[0035] The first output dataset is input into the second-level stacked meta-learner processing module;

[0036] The first-level prediction filtering unit of the second-level stacked meta-learner processing module filters out all the first prediction-label pairs in the first output dataset that correspond to the first prediction-label pair set with the first cross-validation score that is higher than the preset score threshold to form the corresponding second dataset; and trains each meta-learner model according to the second dataset in the KFold training method; and uses Focal Loss+GHM loss as the model training loss function in the model training.

[0037] Preferably, the molecular property prediction processing based on the first ensemble model specifically includes:

[0038] Obtain the molecular data in SMILES format input by the user as the corresponding current molecular data;

[0039] The current molecular data is input into the data preprocessing module, and the data preprocessing module performs anomaly identification on the current molecular data according to the 3 sigma principle to obtain the corresponding first identification result; the first identification result includes anomaly data and non-anomaly data;

[0040] When the first identification result is non-abnormal data, the current molecular data is input into the molecular multimodal characterization processing module. The molecular fingerprint processing unit performs molecular fingerprint feature extraction processing on the current molecular data according to a preset fingerprint format to generate a corresponding first molecular fingerprint feature tensor. The molecular descriptor processing unit performs multi-class descriptor feature extraction processing on the current molecular data according to a preset descriptor type set to generate a corresponding first molecular descriptor feature tensor. The one-dimensional molecular sequence processing unit performs one-dimensional feature extraction processing on the current molecular data using a preset one-dimensional feature extraction model to generate a corresponding first one-dimensional molecular feature tensor. The two-dimensional molecular graph processing unit performs two-dimensional molecular graph feature extraction processing on the current molecular data using a preset two-dimensional feature extraction model to generate a first two-dimensional molecular graph feature tensor. The three-dimensional molecular conformation processing unit performs three-dimensional molecular graph feature extraction processing on the current molecular data using a preset three-dimensional feature extraction model. The first three-dimensional molecular conformation feature tensor is generated; and the first molecular fingerprint feature tensor, the first molecular descriptor feature tensor, the first one-dimensional molecular feature tensor, the first two-dimensional molecular graph feature tensor, the first three-dimensional molecular conformation feature tensor and the first and third molecular attribute labels are used to form the corresponding second molecular characterization data; the preset fingerprint format includes Morgan fingerprint format, ECFP fingerprint format, MACCS fingerprint format, RDKit fingerprint format and Topological fingerprint format; the descriptor type set includes multiple molecular descriptor types; the one-dimensional feature extraction model includes Transformer model, BERT-like model and K-BERT model; the two-dimensional feature extraction model includes GROVER model, MOLCLR model, KPGT model and HIGNN model; the three-dimensional feature extraction model includes EGNN model, SE(3) equal variation neural network and Uni-Mol model;

[0041] The second molecular representation data is input into the multimodal fusion unit of the first-level stacked base learner processing module. The multimodal fusion unit traverses each first correspondence record in a preset first correspondence table reflecting the correspondence between fused tensors and base learners. During the traversal, the currently traversed first correspondence record is taken as the corresponding current record. The first fused tensor feature range field and the first base learner identifier field of the current record are extracted as the corresponding current feature range and current base learner identifier. The base learner that matches the current base learner identifier is taken as the corresponding current base learner. Multimodal feature fusion is performed on one or more feature tensors in the second molecular representation data that match the current feature range to obtain the corresponding first... A two-molecule fusion feature tensor is generated, and a correspondence is established between the second molecular fusion feature tensor and the current base learner identifier. At the end of the traversal, each obtained second molecular fusion feature tensor is input into its corresponding base learner for molecular attribute prediction processing to obtain the corresponding second attribute prediction tensor. The evaluation unit performs cross-validation scoring on each second attribute prediction tensor to obtain the corresponding second cross-validation score. The first correspondence table includes multiple first correspondence records. The first correspondence record includes a first fusion tensor feature range field and a first base learner identifier field. The first fusion tensor feature range field includes fingerprint feature type, descriptor feature type, one-dimensional feature type, two-dimensional feature type, and three-dimensional feature type.

[0042] The first-level stacked base learner processing module then combines each second cross-validation score and the corresponding second attribute prediction tensor to form the corresponding second output data; and all the obtained second output data form the corresponding second output dataset.

[0043] The second output dataset is input into the second-level stacked meta-learner processing module, and the first-level prediction filtering unit filters out the second attribute prediction tensors corresponding to the second cross-validation scores that are higher than the preset scoring threshold in the second output dataset to form the corresponding first tensor sequence; and each of the meta-learners performs molecular attribute prediction processing on each of the second attribute prediction tensors in the first tensor sequence to obtain the corresponding third attribute prediction tensor; and all the obtained third attribute prediction tensors form the corresponding second tensor sequence.

[0044] The second tensor sequence is input into the second-level prediction averaging unit of the second-level stacked meta-learner processing module. The second-level prediction averaging unit calculates the mean of all the third attribute prediction tensors of the second tensor sequence according to the molecular attribute type dimension to obtain the corresponding fourth attribute prediction tensor. The fourth attribute prediction tensor is then output as the model prediction result of the first ensemble model.

[0045] A second aspect of the present invention provides an apparatus for implementing the processing method of the stacked ensemble model for molecular property prediction as described in the first aspect above, the apparatus comprising: a model building module, a model training module, and a model application module;

[0046] The model building module is used to build an integrated model based on a two-level stacking mechanism, denoted as the corresponding first integrated model;

[0047] The model training module is used to train the first ensemble model based on a preset original dataset;

[0048] The model application module is used to perform molecular property prediction processing based on the first integrated model after the model has been successfully trained.

[0049] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0050] The processor is used to couple with the memory, read and execute instructions in the memory to implement the steps of the method described in the first aspect above;

[0051] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0052] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a computer, cause the computer to perform the instructions described in the first aspect.

[0053] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for processing a stacked ensemble model for molecular property prediction. Based on ensemble model theory, a two-level stacked ensemble model framework is constructed. In the first level, multiple parallel base learners (machine learning models, neural network models, or pre-trained models) are introduced to predict molecular properties of multi-scale molecular representations (molecular fingerprints, molecular descriptors, 1D molecular sequences, 2D molecular diagrams, and 3D molecular conformations), and the prediction results of each base learner in the first level are scored. In the second level, multiple parallel meta-learners (machine learning models) are introduced to further refine the coarse prediction results from the first level output. The final molecular property prediction result is obtained by averaging all refined prediction results. Before applying the ensemble model, it is trained using an ensemble training method. This invention solves the problems of single model structure and single molecular representation scale in conventional schemes. Furthermore, the ensemble model prediction method of stacking models at each level and progressively advancing the two-level models further improves the model's generalization ability and prediction accuracy. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of a processing method for a stacked ensemble model for molecular property prediction provided in Embodiment 1 of the present invention;

[0055] Figure 2 This is a model structure diagram of the first integrated model provided in Embodiment 1 of the present invention;

[0056] Figure 3 This is a module structure diagram of a processing device for a stacked integrated model for molecular property prediction provided in Embodiment 2 of the present invention;

[0057] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0059] Embodiment 1 of the present invention provides a method for processing stacked ensemble models for molecular property prediction, such as... Figure 1 The diagram illustrates a method for processing a stacked ensemble model for molecular property prediction provided in Embodiment 1 of the present invention. This method mainly includes the following steps:

[0060] Step 1: Construct an ensemble model based on a two-level stacking mechanism, denoted as the corresponding first ensemble model;

[0061] Among them, such as Figure 2 As shown in the model structure diagram of the first integrated model provided in Embodiment 1 of the present invention, the first integrated model includes a data preprocessing module, a molecular multimodal characterization processing module, a first-level stacked base learner processing module, and a second-level stacked meta-learner processing module; the molecular multimodal characterization processing module includes a molecular fingerprint processing unit, a molecular descriptor processing unit, a one-dimensional molecular sequence processing unit, a two-dimensional molecular graph processing unit, and a three-dimensional molecular conformation processing unit; the first-level stacked base learner processing module includes a multimodal fusion unit, multiple parallel stacked base learners, and an evaluation unit; the second-level stacked meta-learner processing module includes a first-level prediction screening unit, multiple parallel stacked meta-learners, and a second-level prediction averaging unit;

[0062] The input terminals of the molecular fingerprint processing unit, molecular descriptor processing unit, one-dimensional molecular sequence processing unit, two-dimensional molecular graph processing unit, and three-dimensional molecular conformation processing unit are connected to the data preprocessing module, and their output terminals are connected to the multimodal fusion unit, respectively. Multiple output terminals of the multimodal fusion unit are connected to the input terminals of each base learner. The output terminals of each base learner are connected to the input terminal of the evaluation unit. The output terminal of the evaluation unit is connected to the input terminal of the first-level prediction screening unit. Multiple output terminals of the first-level prediction screening unit are connected to the input terminals of each meta-learner. The output terminals of each meta-learner are connected to the input terminal of the second-level prediction averaging unit. The output terminal of the second-level prediction averaging unit is the model output terminal of the first ensemble model.

[0063] The base learner is a first-level molecular attribute prediction model based on a machine learning model, a neural network model, or a pre-trained model, which takes a fusion feature tensor that integrates multimodal molecular features as input. Among them, the machine learning model and the neural network model include at least the Gradient Boosting Decision Tree (GBDT) model, the Extra Tree model, the Logistic Regression (LR) model, the Multilayer Perceptron (MLP) neural network, and the Support Vector Machine (SVM) model. The pre-trained model includes at least the BERT (Bidirectional Encoder Representation from Transformers) model, the Graph Neural Network (GNN) model, the SE(3) equal variation neural network model, and the Uni-Mol model.

[0064] The meta-learner is a secondary molecular attribute prediction model based on a machine learning model, which takes the tensor of the initial molecular attribute as input; among which, the machine learning model includes GBDT model, extraTree model, LR model, MLP neural network and SVM model.

[0065] Step 2: Train the first ensemble model based on the pre-set original dataset;

[0066] Specifically, this includes: Step 21, inputting the original dataset into the data preprocessing module to perform molecular data anomaly detection processing and label data normalization processing to obtain the corresponding preprocessed dataset;

[0067] The original dataset includes multiple first original data sets; each first original data set includes a first molecular data set in SMILES format and a corresponding first molecular attribute label, the first molecular attribute label including multiple molecular attribute types and their corresponding attribute values; the preprocessed dataset includes multiple first preprocessed data sets; each first preprocessed data set includes a second molecular data set in SMILES format and a corresponding second molecular attribute label, the second molecular attribute label including multiple molecular attribute types and their corresponding attribute values.

[0068] Specifically, this includes: Step 211, inputting the original dataset into the data preprocessing module;

[0069] Step 212: The data preprocessing module identifies anomalous data in all the first raw data according to the 3 sigma principle, and marks the first raw data that is not identified as anomalous data as normal raw data; and extracts the first molecular attribute labels of all normal raw data to form the corresponding normal label set.

[0070] Here, the 3 Sigma principle is a processing method for identifying outliers based on the normal distribution of data. For details, please refer to relevant technical implementation literature. It will not be elaborated here.

[0071] Step 213 involves the data preprocessing module extracting all attribute values ​​of any molecular attribute type from the normal label set to form a corresponding first numerical sequence; calculating the skewness of the first numerical sequence to obtain the corresponding first skewness; and when the first skewness is less than or equal to a set threshold, normalizing the first numerical sequence based on a standard Gaussian transform to obtain a corresponding normalized numerical sequence; and when the first skewness is greater than the set threshold, identifying whether the minimum value of the first numerical sequence is greater than 0. If it is, normalizing the first numerical sequence based on a Box-Cox transform to obtain a corresponding normalized numerical sequence; otherwise, normalizing the first numerical sequence based on a Yeo-Johnson transform to obtain a corresponding normalized numerical sequence; and replacing the corresponding attribute values ​​in the normal label set with the normalized attribute values ​​of the obtained normalized numerical sequences.

[0072] Here, the sequence of attribute values ​​for molecular attribute types can be understood through the following example:

[0073] For example, a normal tag set consists of three first-molecular attribute tags: tag 1 {attribute 1 = value 11, attribute 2 = value 12, attribute 3 = value 13}, tag 2 {attribute 1 = value 21, attribute 2 = value 22, attribute 3 = value 23}, and tag 3 {attribute 1 = value 31, attribute 2 = value 32, attribute 3 = value 33}; then, the first numerical sequence of the molecular attribute type = attribute 1 is {value 11, value 21, value 31}.

[0074] Skewness in the steps is a common statistical metric used to assess the skewness of a sample distribution;

[0075] Step 214: The data preprocessing module takes the first molecular attribute label that has been replaced with the normalized attribute value in the normal label set as the corresponding second molecular attribute label, and takes the first molecular data of the normal original data corresponding to each second molecular attribute label as the corresponding second molecular data. The first preprocessed data is composed of each second molecular data and the corresponding second molecular attribute label. The preprocessed dataset is composed of all the first preprocessed data obtained.

[0076] Here, the preprocessed dataset is actually the dataset after removing outliers from the original dataset and normalizing the attribute labels. This preprocessing method is to improve the accuracy of model training.

[0077] Step 22: Input the preprocessed dataset into the molecular multimodal characterization processing module to perform multimodal molecular characterization recognition processing to generate the corresponding first dataset;

[0078] The first dataset includes multiple first molecular characterization data; the first molecular characterization data includes molecular fingerprint feature tensor, molecular descriptor feature tensor, one-dimensional molecular feature tensor, two-dimensional molecular graph feature tensor, three-dimensional molecular conformation feature tensor, and third molecular attribute labels;

[0079] Specifically, this includes: Step 221, inputting the preprocessed dataset into the molecular multimodal characterization processing module;

[0080] Step 222: The molecular fingerprint processing unit of the molecular multimodal characterization processing module performs molecular fingerprint feature extraction processing on the second molecular data of each first preprocessed data according to the preset fingerprint format to generate the corresponding molecular fingerprint feature tensor.

[0081] The preset fingerprint formats include Morgan fingerprint format, ECFP fingerprint format, MACCS fingerprint format, RDKit fingerprint format and Topological fingerprint format, etc.

[0082] Step 223, and the molecular descriptor processing unit of the molecular multimodal characterization processing module performs multi-class descriptor feature extraction processing on the second molecular data of each first preprocessed data according to the preset descriptor type set to generate the corresponding molecular descriptor feature tensor;

[0083] The descriptor type set includes multiple molecular descriptor types;

[0084] Here, the descriptor type set of this embodiment of the invention includes at least 200 molecular descriptor types, commonly including structural descriptors, topological descriptors, connection descriptors, physicochemical descriptors, etc.

[0085] Step 224, and the one-dimensional molecular sequence processing unit of the molecular multimodal characterization processing module uses a preset one-dimensional feature extraction model to perform one-dimensional feature extraction processing on the second molecular data of each first preprocessed data to generate the corresponding one-dimensional molecular feature tensor.

[0086] Among them, one-dimensional feature extraction models include Transformer model, BERT-like model and K-BERT model;

[0087] Here, the model used for one-dimensional feature extraction in this embodiment of the invention has a unified feature: it is a model that can analyze the weight relationship between each character of the input SMILES format molecular data, which is a molecular sequence composed of strings, and the context. Transformer model, BERT-like model, and K-BERT model are all such models. It should be noted that before training the ensemble model, the above-mentioned one-dimensional feature extraction model needs to be trained independently. In this way, when training the ensemble network, there is no need to train them together, which can reduce the overall training difficulty.

[0088] Step 225, and the two-dimensional molecular graph processing unit of the molecular multimodal characterization processing module uses a preset two-dimensional feature extraction model to perform two-dimensional molecular graph feature extraction processing on the second molecular data of each first preprocessed data to generate the corresponding two-dimensional molecular graph feature tensor.

[0089] Among them, the two-dimensional feature extraction models include the GROVER model, MOLCLR model, KPGT model, and HIGNN model;

[0090] Here, the GROVER model can utilize node-level and edge-level contextual attributes for prediction tasks, as well as self-supervised graph-level motif prediction tasks, to learn rich molecular structure and semantic information from a large amount of unlabeled molecular data; the MOLCLR model can compare positive and negative pairs in molecular graphs and employ atom masking, bond loss, and subgraph removal enhancement strategies; the KPGT model can guide the prediction of randomly masked nodes based on knowledge-oriented pre-training strategies and combined with additional molecular knowledge; the HIGNN model is a hierarchical information graph neural network framework designed to predict molecular features using molecular graph and fragment information. It should be noted that before training the ensemble model, the above two-dimensional feature extraction models need to be trained independently, so that they do not need to be jointly trained when training the ensemble network, which can reduce the overall training difficulty.

[0091] Step 226, and the three-dimensional molecular conformation processing unit of the molecular multimodal characterization processing module uses the preset three-dimensional feature extraction model to perform three-dimensional molecular conformation feature extraction processing on the second molecular data of each first preprocessed data to generate the corresponding three-dimensional molecular conformation feature tensor.

[0092] Among them, the three-dimensional feature extraction models include the EGNN model, the SE(3) equal variation neural network and the Uni-Mol model;

[0093] Here, the EGNN model, SE(3) and other variable neural networks and the Uni-Mol model are all neural network models that can extract features of the 3D conformation of molecules based on 3D structure. It should be noted that before training the integrated model, the above three-dimensional feature extraction models need to be trained separately. In this way, when training the integrated network, there is no need to train them together. This approach can reduce the overall training difficulty.

[0094] Step 227, and the molecular multimodal characterization processing module uses the second molecular attribute labels of each first preprocessed data as the corresponding third molecular attribute labels;

[0095] Step 228, the molecular multimodal characterization processing module combines the molecular fingerprint feature tensor, molecular descriptor feature tensor, one-dimensional molecular feature tensor, two-dimensional molecular graph feature tensor, three-dimensional molecular conformation feature tensor, and third molecular attribute label corresponding to each first preprocessed data to form the corresponding first molecular characterization data; and the obtained first molecular characterization data form the corresponding first dataset;

[0096] Step 23: Input the first dataset into the first-level stacked basis learner processing module to train the first-level model and obtain the corresponding first output dataset;

[0097] The first output dataset includes multiple first base learner output data; the first base learner output data includes multiple first batch output data; the first batch output data includes a first prediction-label pair set and a first cross-validation score; the first prediction-label pair set includes multiple first prediction-label pairs; the first prediction-label pair includes a first attribute prediction tensor and a fourth molecular attribute label.

[0098] Specifically, this includes: Step 231, inputting the first dataset into the first-level stacked base learner processing module;

[0099] Step 232: The multimodal fusion unit of the first-level stacked base learner processing module creates a corresponding training dataset for each base learner, denoted as the first learner dataset; and takes any first molecular representation data from the first dataset as the corresponding current molecular representation data; extracts the molecular fingerprint feature tensor, molecular descriptor feature tensor, one-dimensional molecular feature tensor, two-dimensional molecular graph feature tensor, and three-dimensional molecular conformation feature tensor from the current molecular representation data to form the corresponding current feature tensor sequence, and extracts the third molecular attribute label from the current molecular representation data as the corresponding fourth molecular attribute label; and iterates through each first correspondence record in the preset first correspondence table reflecting the correspondence between the fusion tensor and the base learner, and during the traversal, the current correspondence record is... The first correspondence record in the traversal is taken as the corresponding current record. The first fusion tensor feature range field and the first base learner identifier field of the current record are extracted as the corresponding current feature range and current base learner identifier. The base learner that matches the current base learner identifier is taken as the corresponding current base learner. Multimodal feature fusion is performed on one or more feature tensors in the current feature tensor sequence that match the current feature range to obtain the corresponding first molecular fusion feature tensor. The first molecular fusion feature tensor and the corresponding fourth molecular attribute label are used to form the corresponding first learner data. A correspondence is established between the first learner data and the current base learner identifier. At the end of the traversal, each first learner data obtained is stored in its corresponding first learner dataset.

[0100] The first correspondence table includes multiple first correspondence records; each first correspondence record includes a first fusion tensor feature range field and a first base learner identifier field; the first fusion tensor feature range field includes fingerprint feature type, descriptor feature type, one-dimensional feature type, two-dimensional feature type, and three-dimensional feature type; the first learner dataset corresponds one-to-one with the base learner; the first learner dataset includes multiple first learner datasets; the first learner dataset includes a first molecule fusion feature tensor and a fourth molecule attribute label.

[0101] Here, each first correspondence record actually corresponds to a base learner. The feature range stored in the first fusion tensor feature range field refers to the input feature fusion range of the corresponding base learner. For example, if the feature range has only one fingerprint feature type, descriptor feature type, one-dimensional feature type, two-dimensional feature type, or three-dimensional feature type, it means that the corresponding base learner only takes molecular fingerprint features, molecular descriptor features, one-dimensional molecular features, two-dimensional molecular graph features, or three-dimensional molecular conformation features as input. If the feature range includes fingerprint feature type, descriptor feature type, one-dimensional feature type, and two-dimensional feature type, it means that the corresponding base learner takes a fusion feature as input, and the fusion feature integrates four features: molecular fingerprint features, molecular descriptor features, one-dimensional molecular features, and two-dimensional molecular graph features. The multimodal fusion unit can automatically perform data fusion according to the input feature requirements of each base learner through the first correspondence table without the need for additional manual processing or external intervention mechanisms.

[0102] Step 233 involves the first-level stacked base learner processing module dividing each first learner dataset into two datasets according to a preset ratio: a first training dataset and a first validation dataset. The first training dataset is then further divided into multiple data subsets, denoted as corresponding single-batch training data subsets. Based on each single-batch training data subset, the corresponding base learner is trained batch by batch. In each batch of model training, Focal Loss+GHM is used. The loss function is used as the model training loss function. At the end of each batch of model training, the base learner performs molecular attribute prediction processing on each of the first molecular fusion feature tensors of the first validation dataset to generate the corresponding first attribute prediction tensor. Each first attribute prediction tensor and the corresponding fourth molecular attribute label form the corresponding first prediction-label pair. All the obtained first prediction-label pairs form the corresponding first prediction-label pair set. The evaluation unit performs cross-validation scoring on the first prediction-label pair set to obtain the corresponding first cross-validation score. The first prediction-label pair set and the first cross-validation score form the corresponding first batch of output data. At the end of all batches of model training, all the obtained first batch of output data form the corresponding first base learner output data. The first base learner output data corresponding to all base learners form the corresponding first output dataset.

[0103] Here, in this embodiment of the invention, Focal Loss+GHM loss is used as the model training loss function by default in each batch of model training; however, it is also possible to use only Focal Loss as the model training loss function in the first few batches of training, and then use Focal Loss+GHM loss as the model training loss function in subsequent batches. This adjustment can speed up the model training. In addition, other loss functions can be selected for training based on the classification processing type of the base learner, such as regression calculation type, single classification type, multi-class classification type, etc., and the pre-set data balance rules.

[0104] Furthermore, when training each base learner, an independent computing resource is pre-allocated for each base learner, enabling all base learners to perform synchronous computation based on their respective independent computing resources. The independent computing resources include the independent GPU resources and independent CPU resources of the computing device. The computing device includes computers, servers, cloud servers, and high-performance computing clusters. Here, in the embodiments of the present invention, when allocating independent computing resources and planning parallel tasks, the resource isolation of parallel tasks can be achieved based on a container and image processing mechanism similar to dflow.

[0105] Step 24: Input the first output dataset into the second-level stacked meta-learner processing module for second-level model training;

[0106] Specifically, this includes: step 241, inputting the first output dataset into the second-level stacked meta-learner processing module;

[0107] Step 242, and the first-level prediction filtering unit of the second-level stacked meta-learner processing module filters out all the first prediction-label pairs in the first output dataset corresponding to the first cross-validation scores that are higher than the preset score threshold to form the corresponding second dataset; and trains each meta-learner model according to the second dataset in the KFold training method; and in the model training, Focal Loss+GHM loss is used as the model training loss function.

[0108] Here, since each model in the secondary stacked meta-learner processing module is a conventional machine learning model with an approximate linear structure, and its training method is also quite common, it will not be elaborated on here. In addition, the KFold training method is also a commonly used training method for secondary ensemble models, which can be obtained by consulting publicly available technical literature, and will not be elaborated on here either.

[0109] It should be noted that when training each meta-learner, an independent computing resource is pre-allocated for each meta-learner, enabling all meta-learners to perform synchronous computation based on their respective independent computing resources. Independent computing resources include independent GPU and CPU resources of computing devices; computing devices include computers, servers, cloud servers, and high-performance computing clusters. Here, in this embodiment of the invention, when allocating independent computing resources and planning parallel tasks, resource isolation for parallel tasks can be achieved based on a container and image processing mechanism similar to dflow.

[0110] Step 3: If the model training is successful, then perform molecular property prediction based on the first ensemble model.

[0111] Specifically, this includes: Step 31, obtaining the molecular data in SMILES format input by the user as the corresponding current molecular data;

[0112] Step 32: Input the current molecular data into the data preprocessing module, and the data preprocessing module performs abnormal data identification on the current molecular data according to the 3 sigma principle to obtain the corresponding first identification result;

[0113] The first identification result includes both abnormal and non-abnormal data;

[0114] Step 33: When the first identification result is non-abnormal data, the current molecular data is input into the molecular multimodal characterization processing module, and the molecular fingerprint processing unit performs molecular fingerprint feature extraction processing on the current molecular data according to the preset fingerprint format to generate the corresponding first molecular fingerprint feature tensor; the molecular descriptor processing unit performs multi-class descriptor feature extraction processing on the current molecular data according to the preset descriptor type set to generate the corresponding first molecular descriptor feature tensor; and the one-dimensional molecular sequence processing unit uses the preset one-dimensional feature extraction model to perform one-dimensional feature extraction processing on the current molecular data to generate the corresponding first one-dimensional molecular sequence tensor. The first two-dimensional molecular graph feature tensor is generated by the two-dimensional molecular graph processing unit using a preset two-dimensional feature extraction model to perform two-dimensional molecular graph feature extraction on the current molecular data; the first three-dimensional molecular conformation feature tensor is generated by the three-dimensional molecular conformation feature processing unit using a preset three-dimensional feature extraction model to perform three-dimensional molecular graph feature extraction on the current molecular data; and the first two-dimensional molecular fingerprint feature tensor, the first molecular descriptor feature tensor, the first one-dimensional molecular feature tensor, the first two-dimensional molecular graph feature tensor, the first three-dimensional molecular conformation feature tensor, and the first and third molecular attribute labels are combined to form the corresponding second molecular characterization data.

[0115] Among them, the preset fingerprint formats include Morgan fingerprint format, ECFP fingerprint format, MACCS fingerprint format, RDKit fingerprint format and Topological fingerprint format, etc.; the descriptor type set includes multiple molecular descriptor types; the one-dimensional feature extraction model includes Transformer model, BERT-like model and K-BERT model; the two-dimensional feature extraction model includes GROVER model, MOLCLR model, KPGT model and HIGNN model; the three-dimensional feature extraction model includes EGNN model, SE(3) variable neural network and Uni-Mol model;

[0116] Step 34: The second molecular characterization data is input into the multimodal fusion unit of the first-level stacked base learner processing module. The multimodal fusion unit traverses each first correspondence record in the preset first correspondence table reflecting the correspondence between the fusion tensor and the base learner. During the traversal, the currently traversed first correspondence record is taken as the corresponding current record. The first fusion tensor feature range field and the first base learner identifier field of the current record are extracted as the corresponding current feature range and current base learner identifier. The base learner that matches the current base learner identifier is taken as the corresponding current base learner. Multimodal feature fusion is performed on one or more feature tensors in the second molecular characterization data that match the current feature range to obtain the corresponding second molecular fusion feature tensor, and a correspondence is established between the second molecular fusion feature tensor and the current base learner identifier. At the end of the traversal, each obtained second molecular fusion feature tensor is input into its corresponding base learner for molecular attribute prediction processing to obtain the corresponding second attribute prediction tensor. The evaluation unit performs cross-validation scoring on each second attribute prediction tensor to obtain the corresponding second cross-validation score.

[0117] The first correspondence table includes multiple first correspondence records; each first correspondence record includes a first fusion tensor feature range field and a first base learner identifier field; the first fusion tensor feature range field includes fingerprint feature type, descriptor feature type, one-dimensional feature type, two-dimensional feature type and three-dimensional feature type.

[0118] Here, when applying each base learner, an independent computing resource can be pre-allocated to each base learner for prediction; enabling all base learners to perform synchronous computation based on their respective independent computing resources; independent computing resources include independent GPU resources and independent CPU resources of computing devices; computing devices include computers, servers, cloud servers, and high-performance computing clusters; here, in the embodiment of the present invention, when allocating independent computing resources and planning parallel tasks, resource isolation for parallel tasks can be achieved based on a container and image processing mechanism similar to dflow;

[0119] Step 35, and the first-level stacked base learner processing module combines each second cross-validation score and the corresponding second attribute prediction tensor to form the corresponding second output data; and the obtained second output data forms the corresponding second output dataset;

[0120] Step 36: Input the second output dataset into the second-level stacked meta-learner processing module, and the first-level prediction filtering unit filters out the second attribute prediction tensors corresponding to the second cross-validation scores that are higher than the preset scoring threshold in the second output dataset to form the corresponding first tensor sequence; and each meta-learner performs molecular attribute prediction processing on each second attribute prediction tensor in the first tensor sequence to obtain the corresponding third attribute prediction tensor; and all the obtained third attribute prediction tensors form the corresponding second tensor sequence.

[0121] Here, when applying each meta-learner, an independent computing resource can be pre-allocated to each meta-learner for prediction; enabling all base learners to perform synchronous computation based on their respective independent computing resources; independent computing resources include independent GPU resources and independent CPU resources of computing devices; computing devices include computers, servers, cloud servers, and high-performance computing clusters; here, in the embodiment of the present invention, when allocating independent computing resources and planning parallel tasks, resource isolation for parallel tasks can be achieved based on a container and image processing mechanism similar to dflow;

[0122] Step 37: Input the second tensor sequence into the second-level prediction averaging unit of the second-level stacked meta-learner processing module. The second-level prediction averaging unit calculates the average of all third-attribute prediction tensors of the second tensor sequence according to the molecular attribute type dimension to obtain the corresponding fourth-attribute prediction tensor. The fourth-attribute prediction tensor is then output as the model prediction result of the first ensemble model.

[0123] Figure 3 This is a module structure diagram of a processing device for a stacked integrated model for molecular property prediction provided in Embodiment 2 of the present invention. This device can be a terminal device or server implementing the aforementioned method embodiments, or it can be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiments. For example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 3 As shown, the device includes: a model building module 201, a model training module 202, and a model application module 203.

[0124] The model building module 201 is used to build an integrated model based on a two-level stacking mechanism, denoted as the corresponding first integrated model.

[0125] The model training module 202 is used to train the first ensemble model based on a preset original dataset.

[0126] The model application module 203 is used to perform molecular property prediction processing based on the first ensemble model after the model training is successful.

[0127] The processing device for a stacked ensemble model for molecular property prediction provided in this embodiment of the invention can execute the method steps in the above method embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0128] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing elements; they can be fully implemented in hardware; or some modules can be implemented by processing elements calling software, while others are implemented in hardware. For example, the model building module can be a separate processing element, or it can be integrated into a chip in the above device. Alternatively, it can be stored as program code in the memory of the above device, and its functions can be called and executed by a processing element. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0129] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-on-a-Chip (SOC).

[0130] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the foregoing method embodiments are generated. The computer described above can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The aforementioned computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the aforementioned computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) means. The aforementioned computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0131] Figure 4 This is a schematic diagram of an electronic device provided in Embodiment 3 of the present invention. This electronic device can be a terminal device or server implementing the methods of the aforementioned embodiments, or it can be a terminal device or server connected to the aforementioned terminal device or server implementing the methods of the aforementioned embodiments. Figure 4 As shown, the electronic device may include: a processor 301 (e.g., CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transmission and reception operations of the transceiver 303. The memory 302 may store various instructions for performing various processing functions and implementing the processing steps described in the foregoing embodiments. Preferably, the electronic device involved in the embodiments of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The communication port 306 is used for communication between the electronic device and other peripherals.

[0132] exist Figure 4The system bus 305 mentioned can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, it is represented by only one thick line in the diagram, but this does not indicate that there is only one bus or one type of bus. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write libraries, and read-only libraries). Memory may include Random Access Memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0133] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), graphics processing units (GPUs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0134] It should be noted that the embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to perform the methods and processes provided in the above embodiments.

[0135] This invention also provides a chip for executing instructions, which is used to perform the processing steps described in the foregoing method embodiments.

[0136] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for processing a stacked ensemble model for molecular property prediction. Based on ensemble model theory, a two-level stacked ensemble model framework is constructed. In the first level, multiple parallel base learners (machine learning models, neural network models, or pre-trained models) are introduced to predict molecular properties of multi-scale molecular representations (molecular fingerprints, molecular descriptors, 1D molecular sequences, 2D molecular diagrams, and 3D molecular conformations), and the prediction results of each base learner in the first level are scored. In the second level, multiple parallel meta-learners (machine learning models) are introduced to further refine the coarse prediction results from the first level output. The final molecular property prediction result is obtained by averaging all refined prediction results. Before applying the ensemble model, it is trained using an ensemble training method. This invention solves the problems of single model structure and single molecular representation scale in conventional schemes. Furthermore, the ensemble model prediction method of stacking models at each level and progressively advancing the two-level models further improves the model's generalization ability and prediction accuracy.

[0137] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0138] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0139] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for processing stacked ensemble models for molecular property prediction, characterized in that, The method includes: The ensemble model constructed based on the two-level stacking mechanism is denoted as the corresponding first ensemble model. The first ensemble model is trained based on a pre-set raw dataset; If the model is successfully trained, molecular property prediction processing is performed based on the first ensemble model. The first integrated model includes a data preprocessing module, a molecular multimodal characterization processing module, a first-level stacked base learner processing module, and a second-level stacked meta-learner processing module. The molecular multimodal characterization processing module includes a molecular fingerprint processing unit, a molecular descriptor processing unit, a one-dimensional molecular sequence processing unit, a two-dimensional molecular graph processing unit, and a three-dimensional molecular conformation processing unit. The first-level stacked base learner processing module includes a multimodal fusion unit, multiple parallel stacked base learners, and an evaluation unit. The second-level stacked meta-learner processing module includes a first-level prediction screening unit, multiple parallel stacked meta-learners, and a second-level prediction averaging unit. The input terminals of the molecular fingerprint processing unit, the molecular descriptor processing unit, the one-dimensional molecular sequence processing unit, the two-dimensional molecular graph processing unit, and the three-dimensional molecular conformation processing unit are respectively connected to the data preprocessing module, and their output terminals are respectively connected to the multimodal fusion unit; the multiple output terminals of the multimodal fusion unit are respectively connected to the input terminals of each of the base learners; the output terminals of each of the base learners are connected to the input terminal of the evaluation unit; the output terminal of the evaluation unit is connected to the input terminal of the first-level prediction screening unit; the multiple output terminals of the first-level prediction screening unit are connected to the input terminals of each of the meta-learners; the output terminals of each of the meta-learners are connected to the input terminal of the second-level prediction averaging unit; the output terminal of the second-level prediction averaging unit is the model output terminal of the first ensemble model. The base learner is a first-level molecular attribute prediction model based on a machine learning model, a neural network model, or a pre-trained model, which takes a fused feature tensor that integrates multimodal molecular features as input. The machine learning model includes GBDT model, extraTree model, LR model, MLP neural network and SVM model, and the pre-trained model includes BERT model, graph neural network model, SE(3) equal variation neural network model and Uni-Mol model. The meta-learner is a second-level molecular attribute prediction model based on a machine learning model, which takes a primary screening molecular attribute tensor as input. The machine learning model includes GBDT model, extraTree model, LR model, MLP neural network and SVM model. The step of training the first ensemble model based on a preset original dataset specifically includes: The original dataset is input into the data preprocessing module for molecular data anomaly detection and label data normalization to obtain a corresponding preprocessed dataset. The original dataset contains multiple first original datasets. Each first original dataset includes a first molecular data in SMILES format and a corresponding first molecular attribute label. The first molecular attribute label includes multiple molecular attribute types and their corresponding attribute values. The preprocessed dataset contains multiple first preprocessed datasets. Each first preprocessed dataset includes a second molecular data in SMILES format and a corresponding second molecular attribute label. The second molecular attribute label includes multiple molecular attribute types and their corresponding attribute values. The preprocessed dataset is input into the molecular multimodal characterization processing module for multimodal molecular characterization recognition processing to generate a corresponding first dataset; the first dataset includes multiple first molecular characterization data; the first molecular characterization data includes molecular fingerprint feature tensor, molecular descriptor feature tensor, one-dimensional molecular feature tensor, two-dimensional molecular graph feature tensor, three-dimensional molecular conformation feature tensor and third molecular attribute label; The first dataset is input into the first-level stacked base learner processing module to perform first-level model training to obtain the corresponding first output dataset; the first output dataset includes multiple first base learner output data; the first base learner output data includes multiple first batch output data; the first batch output data includes a first prediction-label pair set and a first cross-validation score; the first prediction-label pair set includes multiple first prediction-label pairs; the first prediction-label pair includes a first attribute prediction tensor and the fourth molecular attribute label; The first output dataset is input into the second-level stacked meta-learner processing module for second-level model training. The step of inputting the first dataset into the first-level stacked base learner processing module for first-level model training to obtain the corresponding first output dataset specifically includes: The first dataset is input into the first-level stacked base learner processing module; The multimodal fusion unit of the first-level stacked base learner processing module creates a corresponding training dataset for each base learner, denoted as the first learner dataset. Any first molecular representation data from the first dataset is taken as the corresponding current molecular representation data. The molecular fingerprint feature tensor, molecular descriptor feature tensor, one-dimensional molecular feature tensor, two-dimensional molecular graph feature tensor, and three-dimensional molecular conformation feature tensor from the current molecular representation data are extracted to form the corresponding current feature tensor sequence. The third molecular attribute label from the current molecular representation data is extracted as the corresponding fourth molecular attribute label. The system iterates through each first correspondence record in a preset first correspondence table reflecting the correspondence between fusion tensor and base learner, taking the currently traversed first correspondence record as the current record. The first fusion tensor feature range field and the first base learner identifier field of the current record are extracted as the corresponding current feature range and current base learner identifier. The base learner identifier that matches the current base learner identifier is then selected as the current base learner identifier. The learner serves as the corresponding current base learner and performs multimodal feature fusion on one or more feature tensors in the current feature tensor sequence that match the current feature range to obtain the corresponding first molecular fusion feature tensor. The first molecular fusion feature tensor and the corresponding fourth molecular attribute label constitute the corresponding first learner data, and a correspondence is established between the first learner data and the current base learner identifier. At the end of the traversal, each obtained first learner data is stored in its corresponding first learner dataset. The first correspondence table includes multiple first correspondence records. Each first correspondence record includes a first fusion tensor feature range field and a first base learner identifier field. The first fusion tensor feature range field includes fingerprint feature type, descriptor feature type, one-dimensional feature type, two-dimensional feature type, and three-dimensional feature type. The first learner dataset corresponds one-to-one with the base learner. The first learner dataset includes multiple first learner data sets. The first learner data includes the first molecular fusion feature tensor and the fourth molecular attribute label. The first-level stacked base learner processing module divides each of the first learner datasets into two datasets according to a preset ratio: a first training dataset and a first validation dataset. The first training dataset is then divided into multiple data subsets, denoted as corresponding single-batch training data subsets. Based on each single-batch training data subset, the corresponding base learner is trained batch by batch. In each batch of model training, Focal... Loss+GHMloss is used as the model training loss function; and at the end of each batch of model training, the base learner is used to perform molecular attribute prediction processing on each of the first molecular fusion feature tensors of the first validation dataset to generate the corresponding first attribute prediction tensor, and each of the first attribute prediction tensors and the corresponding fourth molecular attribute label forms the corresponding first prediction-label pair, and all the obtained first prediction-label pairs form the corresponding first prediction-label pair set, and the evaluation unit performs cross-validation scoring on the first prediction-label pair set to obtain the corresponding first cross-validation score, and the first prediction-label pair set and the first cross-validation score form the corresponding first batch output data; and at the end of all batches of model training, all the obtained first batch output data form the corresponding first base learner output data; and the first base learner output data corresponding to all the base learners form the corresponding first output dataset. The molecular property prediction process based on the first ensemble model specifically includes: Obtain the molecular data in SMILES format input by the user as the corresponding current molecular data; The current molecular data is input into the data preprocessing module, and the data preprocessing module performs anomaly identification on the current molecular data according to the 3 sigma principle to obtain the corresponding first identification result; the first identification result includes anomaly data and non-anomaly data; When the first identification result is non-abnormal data, the current molecular data is input into the molecular multimodal characterization processing module. The molecular fingerprint processing unit performs molecular fingerprint feature extraction processing on the current molecular data according to a preset fingerprint format to generate a corresponding first molecular fingerprint feature tensor. The molecular descriptor processing unit performs multi-class descriptor feature extraction processing on the current molecular data according to a preset descriptor type set to generate a corresponding first molecular descriptor feature tensor. The one-dimensional molecular sequence processing unit performs one-dimensional feature extraction processing on the current molecular data using a preset one-dimensional feature extraction model to generate a corresponding first one-dimensional molecular feature tensor. The two-dimensional molecular graph processing unit performs two-dimensional molecular graph feature extraction processing on the current molecular data using a preset two-dimensional feature extraction model to generate a first two-dimensional molecular graph feature tensor. The three-dimensional molecular conformation processing unit performs three-dimensional molecular graph feature extraction processing on the current molecular data using a preset three-dimensional feature extraction model. The first three-dimensional molecular conformation feature tensor is generated; and the first molecular fingerprint feature tensor, the first molecular descriptor feature tensor, the first one-dimensional molecular feature tensor, the first two-dimensional molecular graph feature tensor, the first three-dimensional molecular conformation feature tensor and the first and third molecular attribute labels are used to form the corresponding second molecular characterization data; the preset fingerprint format includes Morgan fingerprint format, ECFP fingerprint format, MACCS fingerprint format, RDKit fingerprint format and Topological fingerprint format; the descriptor type set includes multiple molecular descriptor types; the one-dimensional feature extraction model includes Transformer model, BERT-like model and K-BERT model; the two-dimensional feature extraction model includes GROVER model, MOLCLR model, KPGT model and HIGNN model; the three-dimensional feature extraction model includes EGNN model, SE(3) equal variation neural network and Uni-Mol model; The second molecular representation data is input into the multimodal fusion unit of the first-level stacked base learner processing module. The multimodal fusion unit traverses each first correspondence record in a preset first correspondence table reflecting the correspondence between fused tensors and base learners. During the traversal, the currently traversed first correspondence record is taken as the corresponding current record. The first fused tensor feature range field and the first base learner identifier field of the current record are extracted as the corresponding current feature range and current base learner identifier. The base learner that matches the current base learner identifier is taken as the corresponding current base learner. Multimodal feature fusion is performed on one or more feature tensors in the second molecular representation data that match the current feature range to obtain the corresponding first... A two-molecule fusion feature tensor is generated, and a correspondence is established between the second molecular fusion feature tensor and the current base learner identifier. At the end of the traversal, each obtained second molecular fusion feature tensor is input into its corresponding base learner for molecular attribute prediction processing to obtain the corresponding second attribute prediction tensor. The evaluation unit performs cross-validation scoring on each second attribute prediction tensor to obtain the corresponding second cross-validation score. The first correspondence table includes multiple first correspondence records. The first correspondence record includes a first fusion tensor feature range field and a first base learner identifier field. The first fusion tensor feature range field includes fingerprint feature type, descriptor feature type, one-dimensional feature type, two-dimensional feature type, and three-dimensional feature type. The first-level stacked base learner processing module then combines each of the second cross-validation scores and the corresponding second attribute prediction tensors to form the corresponding second output data; and all the obtained second output data form the corresponding second output dataset. The second output dataset is input into the second-level stacked meta-learner processing module, and the first-level prediction filtering unit filters out the second attribute prediction tensors corresponding to the second cross-validation scores that are higher than the preset scoring threshold in the second output dataset to form the corresponding first tensor sequence; and each of the meta-learners performs molecular attribute prediction processing on each of the second attribute prediction tensors in the first tensor sequence to obtain the corresponding third attribute prediction tensor; and all the obtained third attribute prediction tensors form the corresponding second tensor sequence. The second tensor sequence is input into the second-level prediction averaging unit of the second-level stacked meta-learner processing module. The second-level prediction averaging unit calculates the mean of all the third attribute prediction tensors of the second tensor sequence according to the molecular attribute type dimension to obtain the corresponding fourth attribute prediction tensor. The fourth attribute prediction tensor is then output as the model prediction result of the first ensemble model.

2. The processing method for stacked ensemble models for molecular property prediction according to claim 1, characterized in that, The step of inputting the original dataset into the data preprocessing module for molecular data anomaly detection and label data normalization to obtain the corresponding preprocessed dataset specifically includes: The original dataset is input into the data preprocessing module; The data preprocessing module then performs anomaly identification on all the first raw data according to the 3 sigma principle, and marks the first raw data that is not identified as anomaly as normal raw data; and extracts the first molecular attribute labels of all the normal raw data to form a corresponding normal label set; The data preprocessing module extracts all attribute values ​​of any molecular attribute type from the normal label set to form a corresponding first numerical sequence; calculates the skewness of the first numerical sequence to obtain a corresponding first skewness; when the first skewness is less than or equal to a set threshold, normalizes the first numerical sequence based on a standard Gaussian transform to obtain a corresponding normalized numerical sequence; when the first skewness is greater than the set threshold, identifies whether the minimum value of the first numerical sequence is greater than 0; if so, normalizes the first numerical sequence based on a Box-Cox transform to obtain the corresponding normalized numerical sequence; otherwise, normalizes the first numerical sequence based on a Yeo-Johnson transform to obtain the corresponding normalized numerical sequence; and replaces the corresponding attribute values ​​in the normal label set with the normalized attribute values ​​of the obtained normalized numerical sequences. The data preprocessing module takes each of the first molecular attribute labels that have been replaced with normalized attribute values ​​in the normal label set as the corresponding second molecular attribute label, and takes the first molecular data of the normal original data corresponding to each second molecular attribute label as the corresponding second molecular data. The first preprocessed data is composed of each second molecular data and the corresponding second molecular attribute label. The preprocessed dataset is composed of all the first preprocessed data obtained.

3. The processing method for stacked ensemble models for molecular property prediction according to claim 1, characterized in that, The step of inputting the preprocessed dataset into the molecular multimodal characterization processing module for multimodal molecular characterization recognition processing to generate the corresponding first dataset specifically includes: The preprocessed dataset is input into the molecular multimodal characterization processing module; The molecular fingerprint processing unit of the molecular multimodal characterization processing module performs molecular fingerprint feature extraction processing on the second molecular data of each of the first preprocessed data according to a preset fingerprint format to generate the corresponding molecular fingerprint feature tensor; the preset fingerprint format includes Morgan fingerprint format, ECFP fingerprint format, MACCS fingerprint format, RDKit fingerprint format and Topological fingerprint format; The molecular descriptor processing unit of the molecular multimodal characterization processing module performs multi-class descriptor feature extraction processing on the second molecular data of each of the first preprocessed data according to a preset descriptor type set to generate the corresponding molecular descriptor feature tensor; the descriptor type set includes multiple molecular descriptor types; The one-dimensional molecular sequence processing unit of the molecular multimodal characterization processing module uses a preset one-dimensional feature extraction model to perform one-dimensional feature extraction processing on the second molecular data of each of the first preprocessed data to generate the corresponding one-dimensional molecular feature tensor; the one-dimensional feature extraction model includes the Transformer model, the BERT-like model and the K-BERT model; The two-dimensional molecular graph processing unit of the molecular multimodal characterization processing module uses a preset two-dimensional feature extraction model to perform two-dimensional molecular graph feature extraction processing on the second molecular data of each of the first preprocessed data to generate the corresponding two-dimensional molecular graph feature tensor; the two-dimensional feature extraction model includes the GROVER model, MOLCLR model, KPGT model and HIGNN model; The three-dimensional molecular conformation processing unit of the molecular multimodal characterization processing module uses a preset three-dimensional feature extraction model to perform three-dimensional molecular conformation feature extraction processing on the second molecular data of each of the first preprocessed data to generate the corresponding three-dimensional molecular conformation feature tensor; the three-dimensional feature extraction model includes EGNN model, SE(3) equal variation neural network and Uni-Mol model; The molecular multimodal characterization processing module then uses the second molecular attribute label of each of the first preprocessed data as the corresponding third molecular attribute label. The molecular multimodal characterization processing module then uses the molecular fingerprint feature tensor, the molecular descriptor feature tensor, the one-dimensional molecular feature tensor, the two-dimensional molecular graph feature tensor, the three-dimensional molecular conformation feature tensor, and the third molecular attribute label corresponding to each of the first preprocessed data to form the corresponding first molecular characterization data; and all the obtained first molecular characterization data form the corresponding first dataset.

4. The processing method for stacked ensemble models for molecular property prediction according to claim 1, characterized in that, The step of inputting the first output dataset into the second-level stacked meta-learner processing module for second-level model training specifically includes: The first output dataset is input into the second-level stacked meta-learner processing module; The first-level prediction filtering unit of the second-level stacked meta-learner processing module filters out all the first prediction-label pairs in the first output dataset that correspond to the first prediction-label pair set with the first cross-validation score that is higher than the preset score threshold to form the corresponding second dataset; and trains each meta-learner model according to the second dataset in the KFold training method; and uses Focal Loss+GHM loss as the model training loss function in the model training.

5. An apparatus for performing a processing method for a stacked ensemble model for molecular property prediction as described in any one of claims 1-4, characterized in that, The device includes: a model building module, a model training module, and a model application module; The model building module is used to build an integrated model based on a two-level stacking mechanism, denoted as the corresponding first integrated model; The model training module is used to train the first ensemble model based on a preset original dataset; The model application module is used to perform molecular property prediction processing based on the first integrated model after the model has been successfully trained.

6. An electronic device, characterized in that, include: Memory, processor, and transceiver; The processor is configured to be coupled to the memory, read and execute instructions in the memory to implement the method according to any one of claims 1-4; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1-4.

Images