Method for predicting composition of river planktonic bacterial community based on artificial neural network autoencoder

Abstract

The present application relates to a kind of river planktonic bacteria community composition prediction method based on artificial neural network auto-encoder, belong to environmental science field.The method is by collecting river planktonic bacteria OTU relative abundance and related water environment, meteorological data etc., form original data set, and carry out pre-processing.Using auto-encoder model, the potential space based on OTU composition and the potential space based on environmental elements are constructed by training set, the difference between the two potential spaces is minimized during training, and OTU composition is obtained from potential space by the same decoder.Using SHAP value to explain the model, understand how each environmental factor affects the model prediction of a single sample, and understand the overall impact of environmental factors on model prediction by averaging the SHAP values of multiple samples.The present application explores the change relationship between various environmental factors and river planktonic bacteria relative abundance from the perspective of data-driven deep learning model.

Description

Technical Field

[0001] This invention belongs to the field of environmental science and relates to a method for predicting the composition of river planktonic bacterial communities based on an artificial neural network autoencoder. Background Technology

[0002] A core objective of microbial ecology is to understand the interactions between Earth's microbiome and environmental factors. Current research on the composition of river planktonic bacterial communities primarily focuses on exploring their relationships with environmental elements, their spatiotemporal characteristics, and key driving factors. To overcome the limitations of in-situ monitoring, researchers have developed various models to predict the composition and dynamic changes of river planktonic bacterial communities. These studies are of great significance for developing ecological management plans and addressing environmental change.

[0003] Traditional predictive models, such as mechanistic-based dynamic models (e.g., the Monod equation and the Lotka-Volterra model) and multiple linear regression models, have limitations in predicting microbial community structure. These limitations primarily stem from insufficient applicability to complex natural communities and the neglect of interactions and nonlinear relationships among environmental factors. Furthermore, natural river systems are influenced by multiple complex factors, including geographical, hydrological, and physicochemical factors, making it difficult for traditional multiple regression analysis to fully capture these intricate relationships.

[0004] Against this backdrop, deep learning (DL), as an advanced machine learning method, has demonstrated significant advantages in handling large-scale data and complex models. Although DL has achieved remarkable results in bioinformatics, such as medical image diagnosis and genome sequence analysis, its application in microbiome research remains relatively limited. Currently, while some studies have utilized DL methods (such as LSTM and CNN) to predict microbial community changes, these methods still lack interpretability when dealing with complex nonlinear relationships and interactions between variables. Therefore, developing a more interpretable, data-driven deep learning model is crucial for improving the accuracy and reliability of predicting river planktonic bacterial community composition. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method for predicting the composition of river planktonic bacterial communities based on artificial neural network autoencoders, providing a simpler and more efficient method for studying the composition of river planktonic bacterial communities, and providing technical support for better understanding the potential impact of environmental changes on planktonic bacteria.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for predicting the composition of river planktonic bacterial communities based on artificial neural network autoencoders includes the following steps:

[0008] Data Acquisition and Preparation: Collect river planktonic bacteria samples and related environmental data, including water physicochemical indicators, meteorological elements, geographical patterns, and human activities. This forms a raw dataset covering indicators such as relative abundance of planktonic bacteria, total nitrogen (TN), total phosphorus (TP), dissolved organic carbon (DOC), chlorophyll (Chla), electrical conductivity (Cond), pH, dissolved oxygen (DO), water temperature (WT), air temperature (AT), precipitation (Pre), elevation (elev), and geographical distance. In large watersheds, to ensure sufficient data, data are collected from different spatial locations, such as the main stream and tributaries, ensuring the total sample size is more than twice the required predicted community composition.

[0009] Data preprocessing: Check data validity and remove erroneous and empty data. Standardize units using methods such as standardization, min-max normalization, mean normalization, or logarithmic normalization. Re-encode discrete data such as time and terrain type using built-in one-hot encoding or other methods.

[0010] Model Construction: The model is divided into training and testing sets. Two encoder models and one decoder model are designed. The first encoder converts planktonic bacterial OTU abundance into a latent space (LSOTU). The second encoder converts environmental features into a latent space (LSEF). The decoder converts the latent space back into planktonic bacterial abundance. The latent spaces of the two encoders are trained to be similar, and the differences are minimized to create a comprehensive latent space (LScomb).

[0011] Model Training and Evaluation: An integrated predictor and decoder were used. The predictor returned LScombs from environmental factors, and the decoder returned the relative abundance of planktonic bacterial OTUs from the LScombs. Multiple loss functions and evaluation metrics, such as Bray-Curtis distance and mean squared error (MSE), were used to optimize and evaluate the model. The model was evaluated using test set data, and scores for each evaluation metric were calculated. The SHAP (Shapley Additive Explanations) method was employed to interpret the model and analyze the impact of each environmental factor on the model's prediction results. The SHAP value was calculated based on the following formula:

[0012]

[0013] Where N is the set of features i, which includes total nitrogen (TN), total phosphorus (TP), dissolved organic carbon (DOC), chlorophyll (Chla), electrical conductivity (Cond), pH, dissolved oxygen (DO), water temperature (WT), air temperature (AT), precipitation (Pre), elevation (elev), and geographic distance. S is the subset of features that do not include features i. i f(f) is the SHAP value of feature i, and f(S) is the predicted output of the model given a subset of features S.

[0014] Model Interpretation and Storage: The SHAP method is used to interpret the model, analyzing the impact of each environmental factor on the prediction results. SHAP values ​​are calculated to provide interpretability of the model. The trained encoder and decoder models are saved for future use. By inputting new environmental factors, the model can predict the distribution of planktonic bacteria in new samples.

[0015] The beneficial effects of this invention are as follows:

[0016] (1) This invention uses a deep learning model, especially an autoencoder, to more accurately predict the composition of river planktonic bacterial communities, which improves the accuracy of prediction compared to traditional mechanism-based dynamic models and multiple linear regression models.

[0017] (2) By using SHAP values ​​to interpret the model, this invention can clearly demonstrate how each environmental element affects the model prediction, thereby enhancing the interpretability of the model and making it more valuable for application in the field of environmental science.

[0018] (3) Through a data-driven approach, this invention is not limited to the prediction of specific environmental factors, but can explore the complex relationship between various environmental factors and the relative abundance of river planktonic bacteria, providing broader technical support for ecological environment monitoring and protection.

[0019] (4) This invention helps to develop more accurate and timely ecological management plans and provides a scientific basis for responding to the short-term and long-term impacts of environmental changes by accurately predicting changes in river planktonic bacterial communities.

[0020] (5) The application of this invention in the field of microbiome research, especially in the prediction of river planktonic bacterial community composition, has promoted the application and development of data-driven deep learning technology in the field of environmental science.

[0021] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0022] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0023] Figure 1 This is a flowchart of the present invention;

[0024] Figure 2 These are the results collected during the training process in a specific embodiment of the present invention;

[0025] Figure 3 The results of testing the model using the test set in a specific embodiment of the present invention;

[0026] Figure 4 This is a global bar chart of the SHAP values ​​of the encoder, decoder, and predictor in a specific embodiment of the present invention; Figure 4 (a) is the encoder SHAP; Figure 4 (b) is the decoder SHAP; Figure 4 (c) is the prediction period SHAP. Detailed Implementation

[0027] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0028] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0029] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0030] Figure 1 This is a flowchart of the present invention. This embodiment uses an autoencoder model based on deep artificial neural networks to predict the composition of river planktonic bacterial communities using environmental factors. The method predicts the planktonic bacterial community in the upper reaches of the Yangtze River, and specifically includes the following steps:

[0031] 1. Compile and organize the OTU data of planktonic bacteria and related hydrological and aquatic environment data from 212 samples collected along the upper reaches of the Yangtze River (from Batang, Sichuan to Yichang, Hubei) between 2015 and 2022 to form the original dataset. Specific indicators include: time (Year, Month), total nitrogen (TN), total phosphorus (TP), dissolved organic carbon (DOC), chlorophyll (Chla), conductivity (Cond), pH, dissolved oxygen (DO), water temperature (WT), stoichiometry (C / P, C / N, N / P), air temperature (AT), precipitation (Pre), elevation (elev), geographical distance (distance from the Three Gorges Dam in this example), and reservoir age (RA). Among them, Year, Month, TN, TP, DOC, Chla, Cond, pH, DO, and WT are environmental variables measured in the raw water of the samples, while AT, Pre, elev, distance, and RA are environmental covariates collected subsequently. The original environmental dataset is shown in Table 1, and the OTU table of planktonic bacteria is shown in Table 2. In this embodiment, the OTU relative abundance table at the phylum level of planktonic bacteria was selected for training.

[0032] Table 1

[0033]

[0034] Table 2

[0035]

[0036]

[0037] 2. The raw data from the Three Gorges Reservoir was preprocessed. Data validity was checked, erroneous and empty data were removed, and discontinuous time data were one-hot encoded. The processed data was then normalized using standard methods. The normalized data is shown in Table 3.

[0038] Table 3

[0039]

[0040] 3. The dataset was divided into a training set (n=148) and a test set (n=64) in a 7:3 ratio. An AE algorithm was used to construct a predictive model for the planktonic bacterial community structure. Common evaluation metrics such as Mean Absolute Error (MAE), Mean Squared Error (MSE), and Mean Absolute Percentage Error (MAPE) were used during model construction. However, these metrics are difficult to interpret, especially under different standardization methods, and are easily affected by small values. Therefore, this invention calculated additional scale-independent metrics, such as Pearson Correlation, which calculates the average correlation between each predicted OTU and the actual OTU, taking the average of all samples; Bray-Curtis Dissimilarity, which calculates the similarity between predicted and actual values, measuring the difference between two samples; and Root Relative Squared Error (RRSE), which evaluates the prediction error of a single OTU, rather than the entire sample. See [link to AE model training documentation]. Figure 2 This includes training and validation losses and metrics; test performance is available in [link to performance documentation]. Figure 3 .

[0041] 4. The SHAP values ​​were used to interpret the model prediction results. The SHAP values ​​of the encoder, decoder, and predictor were calculated separately. Figure 4 This section explains how OTU data is processed by the encoder and transformed into the latent space, how the decoder reconstructs the OTU data from the latent space, and how the predictor uses environmental features to predict the latent space. In the figure, the X-axis represents the average SHAP value, indicating the average impact of each feature on the model output; the Y-axis represents different input features. Figure 4 (a) shows the types of planktonic bacteria. Figure 4 (b) shows the latent features after dimensionality reduction. Figure 4 (c) shows environmental features; in the diagram, "Class" represents the different categories output by the model. Because this is a multi-class classification problem, the model may output probability distributions for multiple categories. Each color corresponds to one category (Class), for example... Figure 4(a) shows 20 categories (from Class 0 to Class 19). In this case, the categories may be different bacterial phyla. The figure ranks the "importance" of the input features, with the influence of features on the model's output gradually decreasing from top to bottom. Year, water temperature, and C / P ratio are the three most important environmental features. The weights of environmental factors in the prediction model reflect, to some extent, the influence of environmental factors on the corresponding prediction targets. For example, biogeographical patterns (geographical distance, elevation, latitude and longitude) have a relatively significant impact on the prediction of planktonic bacterial community composition, further illustrating that spatial variation largely causes changes in planktonic bacterial community composition. This is similar to the results of Liu et al.'s 2018 article "Integrated biogeography of planktonic and sedimentary bacterial communities in the Yangtze River" published in the journal Microbiome.

[0042] 5. Based on the results of the previous step, select the main environmental features and train a planktonic bacterial community prediction model that requires fewer environmental features. In this embodiment, "YEAR", "WT", "Pre", "C / P", and "distance" were selected. The Bray-Curtis dissimilarity was 0.1933 and the Pearson correlation coefficient was 0.9153. Collecting these five types of environmental data for the sample points to be predicted will yield the river planktonic bacterial community structure at the corresponding locations. The results are saved in the corresponding .tsv files for further analysis.

[0043] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for predicting the composition of river planktonic bacterial communities based on artificial neural network autoencoders, characterized by: The method comprises the following steps: Collecting relevant environmental data and biological data; Data preprocessing; Divide the training set and the test set; Use the autoencoder model to construct the latent space of environmental factors and biological community composition, optimize the model and evaluate its performance; Use SHAP value to explain the model and understand the influence of environmental factors on model prediction; Among them, the environmental factors include water environment and meteorological data, and the biological data is the relative abundance data of planktonic bacteria OTU; In the division of the training set and the test set, the first encoder, the second encoder and the decoder are designed, the first encoder takes the planktonic bacteria OTU abundance as the input feature and converts it into the latent space LSOTU; the second encoder takes the environmental factors as the input feature and converts it into the latent space LSEF; the decoder converts the latent space into planktonic bacteria abundance; The first encoder and the second encoder are trained into a predictor, and in the training process, the latent spaces of the two encoders are forced to be similar, that is, as part of the loss function, the difference between the two latent spaces is minimized, and the latent space LScomb of LSOTU is created for environmental variables; The final model integrates a predictor and a decoder, the predictor returns LScomb from environmental factors, and the decoder returns the predicted relative abundance of planktonic bacteria OTU from LScomb.

2. The method of claim 1, wherein the method is based on an artificial neural network autoencoder. The data preprocessing includes checking data validity, removing error data and empty data, unifying dimension, and using standardization, minimum-maximum normalization, mean normalization or logarithmic normalization method.

3. The method of claim 1, wherein the method is based on an artificial neural network autoencoder. The indicators for optimizing the model and evaluating its performance include mean absolute error, mean square error, mean absolute percentage error, Pearson correlation coefficient, Bray-Curtis dissimilarity and root relative squared error.

4. The method of claim 1, wherein the method is based on an artificial neural network autoencoder. The performance evaluation of the model on the test set includes calculating the scores of each evaluation indicator; SHAP method is used to explain the model and analyze the influence of each environmental factor on the model prediction result; the calculation of SHAP value is based on the following formula: where N is the set of all features, S is a subset of features not including feature i, φ i (f) is the SHAP value for feature i, f(S) is the model’s predicted output given the subset of features S. Save the trained encoder and decoder model for subsequent use, and predict the biological community composition of new samples by inputting new environmental factors.

5. The method of claim 1, wherein the method is based on an artificial neural network autoencoder. The environmental factors also include geographical factors, hydrological parameters and physicochemical indicators.

6. The method of claim 4, wherein the method is based on an artificial neural network autoencoder. The evaluation indicators also include scale-independent indicators, specifically Pearson correlation coefficient, Bray-Curtis dissimilarity and root relative squared error RRSE.

Images