Water pollutant molecular property prediction method and system based on pre-training model and multi-source coding feature fusion

Abstract

The invention discloses a pre-training model and multi-source coding feature fusion-based water pollutant molecular property prediction method and system, and relates to the technical field of environmental science and artificial intelligence crossing. Comprising the steps of collecting pollutant data; the collected pollutant data is preprocessed; taking SMILES as a molecular input basis, and extracting molecular features; according to the molecular features, feature vectors are obtained from an encoder and then input to a vector interaction module for feature fusion; and embedding the comprehensive molecules obtained after feature fusion into an expert mixed structure module to obtain comprehensive representation of an expert mechanism, inputting the comprehensive representation of the expert mechanism into a preset prediction head, and completing prediction of molecular properties through the prediction head. According to the method, the molecular language model features, the molecular graph neural network features and the molecular descriptor features are integrated, and feature fusion is carried out by utilizing an attention mechanism and residual connection, so that high-precision prediction of the molecular properties of pollutants is realized.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of environmental science and artificial intelligence, and more specifically to a method and system for predicting the molecular properties of water pollutants based on the fusion of pre-trained models and multi-source coding features. Background Technology

[0002] With industrialization and urbanization, large amounts of organic and emerging pollutants enter water bodies, posing serious environmental and health risks. Traditional pollutant prediction methods rely on experimental detection and molecular simulation; the former is costly and time-consuming, while the latter is computationally complex and lacks generalization ability. In recent years, the application of machine learning and deep learning in cheminformatics has matured, enabling the prediction of physicochemical properties and biological activities through chemical structures, and thereby assessing the recalcitrantness, mobility, and toxicity of pollutants. Simultaneously, water quality anomaly detection can promptly identify sudden pollution events and abnormal fluctuations, such as illegal discharges, pipeline leaks, or short-term high concentrations of emerging pollutants. Combining prediction and anomaly detection not only allows for the early identification of high-risk pollutants but also enables rapid response during and after events, ensuring drinking water safety and ecosystem stability, and forming a multi-layered environmental risk early warning and control system. However, existing methods generally rely on only a single type of feature representation, such as molecular fingerprints or molecular graphs, which struggles to comprehensively capture the structural and semantic information of pollutant molecules, resulting in room for improvement in prediction accuracy and generalization ability.

[0003] Therefore, proposing a method and system for predicting the molecular properties of water pollutants based on the fusion of pre-trained models and multi-source coding features to address the difficulties in existing technologies is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, the present invention provides a method and system for predicting the molecular properties of water pollutants based on the fusion of pre-trained models and multi-source coding features. By integrating molecular language model features, molecular graph neural network features, and molecular descriptor features, and using attention mechanisms and residual connections for feature fusion, high-precision prediction of the molecular properties of pollutants can be achieved.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for predicting the molecular properties of water pollutants based on the fusion of pre-trained models and multi-source coding features includes the following steps: S1. Data Collection: Collect data on pollutants with physicochemical properties or toxicological labels; S2. Data preprocessing: The collected pollutant data is preprocessed to obtain preprocessed pollutant data; S3. Molecular Input and Feature Extraction: Using SMILES as the molecular input basis, extract molecular features; S4. Feature Fusion: Based on the molecular features, the feature vectors are obtained from the encoder and then input into the vector interaction module for feature fusion. S5. Prediction Output: The comprehensive molecule obtained after feature fusion is embedded into the expert hybrid structure module to obtain the comprehensive representation of the expert mechanism. The comprehensive representation of the expert mechanism is input into the preset prediction head, and the prediction of molecular properties is completed through the prediction head.

[0006] Optionally, S3 uses SMILES as the molecular input basis and employs three methods to extract molecular features: First, it extracts molecular fingerprints and physicochemical descriptors based on RDKit to form sparse binary or counting vectors, obtains local substructure patterns, and further obtains low-dimensional numerical feature vectors; second, it inputs SMILES into the pre-trained molecular language model molformer to obtain contextual semantic features; and third, it transforms the molecular structure into a molecular graph and then extracts the graph structure embedding through the pre-trained graph neural network molE.

[0007] Optionally, in S4, the specific steps for obtaining feature vectors from the encoder based on molecular characteristics and then inputting them into the vector interaction module for feature fusion are as follows: The vector interaction module first performs linear transformation and normalization on feature vectors of different dimensions to make the feature vectors in the same representation space. Then, it introduces a multi-head attention mechanism and residual connection to model the relationship between features of different modalities. The model adjusts the attention to each feature according to the preset molecular properties to predict the task target, so as to complete feature fusion.

[0008] Optionally, when the vector interaction module performs linear transformations on feature vectors of different dimensions, it adjusts the dimensions of the feature vectors by setting a learnable weight matrix; when performing normalization, it adopts layer normalization or batch normalization.

[0009] Optionally, the preset molecular property prediction task objectives include any one of the following: physicochemical property prediction, toxicity prediction, and drug activity prediction.

[0010] Optionally, in S5, the comprehensive molecule obtained after feature fusion is embedded into the expert hybrid structure module to obtain a comprehensive representation of the expert mechanism. The comprehensive representation of the expert mechanism is then input into a preset prediction head, and the specific content of the prediction of molecular properties by the prediction head is as follows: A gated network is introduced into the expert hybrid structure module. The gated network outputs a set of probability distributions based on the feature distribution of the input comprehensive molecular embedding. Based on the probability distributions, several experts are automatically selected or activated by weighting. The outputs of the activated experts are fused by weighted summation or concatenation to obtain a comprehensive representation of the expert mechanism; The comprehensive representation of the expert mechanism is input into the preset prediction head, and the prediction head completes the prediction of molecular properties.

[0011] Optionally, a sparse activation mechanism can be used in the expert hybrid structure module to avoid all experts working at the same time.

[0012] A water pollutant molecular property prediction system based on the fusion of a pre-trained model and multi-source coding features, applying any one of the above-mentioned water pollutant molecular property prediction methods based on the fusion of a pre-trained model and multi-source coding features, includes: a data collection module, a data preprocessing module, a molecular input and feature extraction module, a feature fusion module, and a prediction output module; The data collection module, connected to the input of the data preprocessing module, is used to collect pollutant data with physicochemical properties or toxicological labels; The data preprocessing module is connected to the input end of the molecular input and feature extraction module. It is used to preprocess the collected pollutant data to obtain preprocessed pollutant data. The molecular input and feature extraction module is connected to the input end of the feature fusion module and is used to extract molecular features based on SMILES as the molecular input. The feature fusion module is connected to the input of the prediction output module. It is used to obtain feature vectors from the encoder based on molecular features and then input them to the vector interaction module for feature fusion. The prediction output module is used to embed the comprehensive molecule obtained after feature fusion into the expert hybrid structure module to obtain a comprehensive representation of the expert mechanism. The comprehensive representation of the expert mechanism is then input into a preset prediction head, which is used to predict the molecular properties.

[0013] As can be seen from the above technical solution, compared with the prior art, the present invention provides a method and system for predicting the molecular properties of water pollutants based on the fusion of pre-trained models and multi-source coding features, which has the following beneficial effects: This invention achieves high-precision prediction of the molecular properties of pollutants by integrating features from molecular language models, molecular graph neural networks, and molecular descriptors, and by using attention mechanisms and residual connections for feature fusion. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0015] Figure 1 A flowchart of a method for predicting the molecular properties of water pollutants based on the fusion of pre-trained models and multi-source coding features provided by this invention; Figure 2 A schematic diagram of the vector interaction module structure is provided for this invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] SMILES is a standardized string format used in the field of chemistry to represent molecular structures. It stands for Simplified Molecular-Input Line-Entry System. It uses specific combinations of letters, numbers and symbols to convert complex molecular structures (such as atom types, chemical bonds, ring structures, branches, etc.) into a text form that can be directly input into a computer, facilitating the storage, transmission and computer processing of molecular data.

[0018] See Figure 1 As shown, this invention discloses a method for predicting the molecular properties of water pollutants based on the fusion of a pre-trained model and multi-source coding features, comprising the following steps: S1. Data Collection: Collect data on pollutants with physicochemical properties or toxicological labels; S2. Data preprocessing: The collected pollutant data is preprocessed to obtain preprocessed pollutant data; S3. Molecular Input and Feature Extraction: Using SMILES as the molecular input basis, extract molecular features; S4. Feature Fusion: Based on the molecular features, the feature vectors are obtained from the encoder and then input into the vector interaction module for feature fusion. S5. Prediction Output: The comprehensive molecule obtained after feature fusion is embedded into the expert hybrid structure module to obtain the comprehensive representation of the expert mechanism. The comprehensive representation of the expert mechanism is input into the preset prediction head, and the prediction of molecular properties is completed through the prediction head.

[0019] Furthermore, S3 uses SMILES as the molecular input basis and employs three methods to extract molecular features: First, it extracts molecular fingerprints and physicochemical descriptors based on RDKit to form sparse binary or counting vectors, obtains local substructure patterns, and further obtains low-dimensional numerical feature vectors; second, it inputs SMILES into the pre-trained molecular language model molformer to obtain contextual semantic features; and third, it transforms the molecular structure into a molecular graph and then extracts the graph structure embedding through the pre-trained graph neural network molE.

[0020] Furthermore, in S4, the specific steps for obtaining feature vectors from the encoder based on molecular characteristics and then inputting them into the vector interaction module for feature fusion are as follows: The vector interaction module first performs linear transformation and normalization on feature vectors of different dimensions to make the feature vectors in the same representation space. Then, it introduces a multi-head attention mechanism and residual connection to model the relationship between features of different modalities. The model adjusts the attention to each feature according to the preset molecular properties to predict the task target, so as to complete feature fusion.

[0021] Furthermore, when the vector interaction module performs linear transformations on feature vectors of different dimensions, it adjusts the dimensions of the feature vectors by setting a learnable weight matrix; when performing normalization, it adopts layer normalization or batch normalization.

[0022] Furthermore, the pre-defined molecular property prediction task objectives include any one of the following: physicochemical property prediction, toxicity prediction, and drug activity prediction.

[0023] Furthermore, in S5, the comprehensive molecule obtained after feature fusion is embedded into the expert hybrid structure module to obtain a comprehensive representation of the expert mechanism. This comprehensive representation is then input into a preset prediction head, and the prediction of molecular properties is performed using this prediction head. A gated network is introduced into the expert hybrid structure module. The gated network outputs a set of probability distributions based on the feature distribution of the input comprehensive molecular embedding. Based on the probability distributions, several experts are automatically selected or activated by weighting. The outputs of the activated experts are fused by weighted summation or concatenation to obtain a comprehensive representation of the expert mechanism; The comprehensive representation of the expert mechanism is input into the preset prediction head, and the prediction head completes the prediction of molecular properties.

[0024] Furthermore, it also includes employing a sparse activation mechanism in the expert hybrid structure module to avoid all experts working simultaneously.

[0025] In one specific embodiment, the following is included: Data Collection and Preprocessing: Collect contaminant data for toxicology labels from publicly available literature and experimental reports. Prioritize compounds with CAS numbers to ensure the uniqueness and traceability of molecular identities. Retrieve corresponding SMILES expressions based on CAS numbers in PubChem or other databases. Standardize molecular structures, including unifying aromaticity representations; handling explicit / implicit hydrogen atoms; standardizing charge and valence states; and eliminating stereoisomer ambiguities in the structure. Remove duplicate entries through standardized SMILES comparisons. Check for conflicting labels for the same compound (e.g., significant differences in physicochemical property data), confirming or deleting inconsistent entries if necessary through literature review. Remove inorganic substances without carbon skeletons (e.g., salts, metal ion compounds). Remove small molecule gases (e.g., C2O2). N Non-target contaminants such as common solvents are excluded to ensure that the dataset is dominated by organic contaminants. The cleaned molecular data are uniformly saved as a structured table corresponding to SMILES and labels. The dataset is then randomly divided according to a certain proportion (e.g., 70% for training, 15% for validation, and 15% for testing) to ensure fairness and reproducibility in model training and evaluation.

[0026] Molecular Input and Feature Extraction: Using SMILES as the molecular input basis, three methods are employed to extract molecular features: First, based on RDKit, molecular fingerprints and physicochemical descriptors are extracted to form sparse binary or counting vectors, obtaining local substructure patterns and further obtaining low-dimensional numerical feature vectors; second, SMILES is input into the pre-trained molecular language model molformer to obtain contextual semantic features; third, the molecular structure is transformed into a molecular graph, and then the graph structure embedding is extracted through the pre-trained graph neural network molE. MolFormer takes the SMILES sequence as input and uses a Transformer Encoder structure to learn the contextual semantic representation of the molecule. This framework is trained using classic masking. In this embodiment, its inference part is used to derive the semantic vector, specifically by tokenizing the SMILES string, with each token t... i Mapped to a 768-dimensional embedding;

[0027] in, For word embedding matrix, Embedded for spatial location; In each Transformer layer, a multi-head attention mechanism (MHA) and a feedforward network (FFN) are applied.

[0028] in, ;

[0029] Finally, the token-level molecular semantic representation and the global pooling vector are input to the hidden layer vector interaction module, with shapes of (32, 512, 768) and (32, 768), respectively.

[0030] Two MolEs are based on molecular diagrams ,in A collection of atoms For chemical bonds, each atomic node Assign initial feature vectors ; In the During layer message passing:

[0031] in, Indicates the first Layer nodes express, Represents a node The neighborhood group, These are learnable parameters, used to control the weights of the self-loop. Indicates the first Multilayer perceptron; The output vector has dimensions of (32, 128, 768). Since the number of nodes varies in each node graph, the maximum number of nodes in this dataset is chosen. Each node is embedded and padded to a length of 128 using a zero-vector method. Any extra positions are masked by constructing a mask to shield the subsequent multi-head attention calculation. The graph-level molecular embedding is obtained by the readout function and has a shape of (32, 768).

[0032] The three-molecule fingerprint is generated into a 2048-dimensional vector using ECFP4, and the iterative update formula is as follows:

[0033] in, Represents atoms In iteration radius The sign below, It is the type of the key. If a substructure appears, the corresponding position is marked as 1;

[0034] After a linear transformation, it is mapped to a 768-dimensional vector and input to the vector interaction module. The vector with shape (32, 1, 768) enters the attention calculation and (32, 768) is directly input to the expert hybrid structure module.

[0035] Feature fusion: such as Figure 2 As shown, the vector interaction module first performs linear transformation and normalization on feature vectors of different dimensions to make the feature vectors in the same representation space. Then, it introduces a multi-head attention mechanism and residual connection to model the relationship between features of different modalities. The model adjusts the attention to each feature according to the preset molecular properties to predict the task target, so as to complete feature fusion. The vector interaction module introduces a multi-head attention mechanism and residual connections, by concatenating vectors. Construct fused inputs, and construct attention masks for semantic vectors, graph vectors, and fingerprint vectors. The specific calculation logic is as follows: Single-head attention mechanism ; Multi-head attention mechanism ; in, ; The calculation logic for residual connectivity is as follows: .

[0036] Predictive Output: A gating network is introduced into the expert hybrid structure module. The gating network outputs a set of probability distributions based on the feature distribution of the input comprehensive molecular embedding. Several experts are automatically selected or activated based on the probability distributions. The outputs of the activated experts are fused by weighted summation or concatenation to obtain a comprehensive representation of the expert mechanism. The comprehensive representation of the expert mechanism is input into a preset prediction head, which is used to predict molecular properties. The vectors from the hidden layer vector interaction module are concatenated and then input into the expert mechanism module. The calculation logic of the expert mechanism module is as follows:

[0037] in, The weights are given by the gating network, satisfying... ; Top K sparse activation is used to facilitate the model in extracting useful features for prediction, thereby reducing the computational cost of inference.

[0038] in, It is the model input; It refers to the number of experts. For expert indexing; It is the TopK expert index set, where k is the number of active experts; It is the gating parameter vector corresponding to the i-th expert; The numerator is the score given by the i-th expert; j is the score given by the i-th expert in the normalized denominator corresponding to the selected set S. k The summation index of all experts j is calculated using softmax normalization; Finally, the classification objective is achieved through the MLP prediction head:

[0039] in, It is the MLP Hidden representation of layers; It is an activation function; , It is the first Layer weights and biases; It is the number of hidden layers.

[0040] A water pollutant molecular property prediction system based on the fusion of a pre-trained model and multi-source coding features, applying any one of the above-mentioned water pollutant molecular property prediction methods based on the fusion of a pre-trained model and multi-source coding features, includes: a data collection module, a data preprocessing module, a molecular input and feature extraction module, a feature fusion module, and a prediction output module; The data collection module, connected to the input of the data preprocessing module, is used to collect pollutant data with physicochemical properties or toxicological labels; The data preprocessing module is connected to the input end of the molecular input and feature extraction module. It is used to preprocess the collected pollutant data to obtain preprocessed pollutant data. The molecular input and feature extraction module is connected to the input end of the feature fusion module and is used to extract molecular features based on SMILES as the molecular input. The feature fusion module is connected to the input of the prediction output module. It is used to obtain feature vectors from the encoder based on molecular features and then input them to the vector interaction module for feature fusion. The prediction output module is used to embed the comprehensive molecule obtained after feature fusion into the expert hybrid structure module to obtain a comprehensive representation of the expert mechanism. The comprehensive representation of the expert mechanism is then input into a preset prediction head, which is used to predict the molecular properties.

[0041] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0042] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for predicting the molecular properties of water pollutants based on the fusion of a pre-trained model and multi-source coding features, characterized in that, Includes the following steps: S1. Data Collection: Collect data on pollutants with physicochemical properties or toxicological labels; S2. Data preprocessing: The collected pollutant data is preprocessed to obtain preprocessed pollutant data; S3. Molecular Input and Feature Extraction: Using SMILES as the molecular input basis, extract molecular features; S4. Feature Fusion: Based on the molecular features, the feature vectors are obtained from the encoder and then input into the vector interaction module for feature fusion. S5. Prediction Output: The comprehensive molecule obtained after feature fusion is embedded into the expert hybrid structure module to obtain the comprehensive representation of the expert mechanism. The comprehensive representation of the expert mechanism is input into the preset prediction head, and the prediction of molecular properties is completed through the prediction head.

2. The method for predicting the molecular properties of water pollutants based on the fusion of a pre-trained model and multi-source coding features according to claim 1, characterized in that, S3 uses SMILES as the molecular input basis and employs three methods to extract molecular features: First, it extracts molecular fingerprints and physicochemical descriptors based on RDKit to form sparse binary or counting vectors, obtains local substructure patterns, and further obtains low-dimensional numerical feature vectors; Second, it inputs SMILES into the pre-trained molecular language model molformer to obtain contextual semantic features. Third, the molecular structure is converted into a molecular graph, and then the graph structure is extracted and embedded through a pre-trained graph neural network molE.

3. The method for predicting the molecular properties of water pollutants based on the fusion of a pre-trained model and multi-source coding features according to claim 1, characterized in that, In S4, the specific steps for obtaining feature vectors from the encoder based on molecular characteristics and then inputting them into the vector interaction module for feature fusion are as follows: The vector interaction module first performs linear transformation and normalization on feature vectors of different dimensions to make the feature vectors in the same representation space. Then, it introduces a multi-head attention mechanism and residual connection to model the relationship between features of different modalities. The model adjusts the attention to each feature according to the preset molecular properties to predict the task target, so as to complete feature fusion.

4. The method for predicting the molecular properties of water pollutants based on the fusion of a pre-trained model and multi-source coding features according to claim 3, characterized in that, When the vector interaction module performs linear transformations on feature vectors of different dimensions, it adjusts the dimensions of the feature vectors by setting a learnable weight matrix; when performing normalization, it adopts layer normalization or batch normalization.

5. The method for predicting the molecular properties of water pollutants based on the fusion of a pre-trained model and multi-source coding features according to claim 3, characterized in that, The preset molecular property prediction task objectives include any one of the following: physicochemical property prediction, toxicity prediction, and drug activity prediction.

6. The method for predicting the molecular properties of water pollutants based on the fusion of a pre-trained model and multi-source coding features according to claim 1, characterized in that, In S5, the comprehensive molecule obtained after feature fusion is embedded into the expert hybrid structure module to obtain a comprehensive representation of the expert mechanism. The comprehensive representation of the expert mechanism is then input into a preset prediction head, and the specific content of the prediction of molecular properties by the prediction head is as follows: A gated network is introduced into the expert hybrid structure module. The gated network outputs a set of probability distributions based on the feature distribution of the input comprehensive molecular embedding. Based on the probability distributions, several experts are automatically selected or activated by weighting. The outputs of the activated experts are fused by weighted summation or concatenation to obtain a comprehensive representation of the expert mechanism; The comprehensive representation of the expert mechanism is input into the preset prediction head, and the prediction head completes the prediction of molecular properties.

7. The method for predicting the molecular properties of water pollutants based on the fusion of a pre-trained model and multi-source coding features according to claim 6, characterized in that, It also includes the use of a sparse activation mechanism in the expert hybrid structure module to avoid all experts working at the same time.

8. A system for predicting the molecular properties of water pollutants based on the fusion of a pre-trained model and multi-source coding features, characterized in that, The method for predicting the molecular properties of water pollutants based on the fusion of a pre-trained model and multi-source coding features according to any one of claims 1-7 includes: a data collection module, a data preprocessing module, a molecular input and feature extraction module, a feature fusion module, and a prediction output module; The data collection module, connected to the input of the data preprocessing module, is used to collect pollutant data with physicochemical properties or toxicological labels; The data preprocessing module is connected to the input end of the molecular input and feature extraction module. It is used to preprocess the collected pollutant data to obtain preprocessed pollutant data. The molecular input and feature extraction module is connected to the input end of the feature fusion module and is used to extract molecular features based on SMILES as the molecular input. The feature fusion module is connected to the input of the prediction output module. It is used to obtain feature vectors from the encoder based on molecular features and then input them to the vector interaction module for feature fusion. The prediction output module is used to embed the comprehensive molecule obtained after feature fusion into the expert hybrid structure module to obtain a comprehensive representation of the expert mechanism. The comprehensive representation of the expert mechanism is then input into a preset prediction head, which is used to predict the molecular properties.

Images