Environmental microbial community state assessment method based on unsupervised clustering

Abstract

The invention discloses an environmental microbial community state evaluation method based on unsupervised clustering, and the specific implementation mode is as follows: obtaining metagenome sequencing data in an environmental sample, and carrying out microbial species annotation and abundance calculation; based on abundance data, sample microbial species richness and Shannon diversity indexes are calculated, microbial functional gene annotation and abundance calculation are carried out, standardization processing, principal component analysis, Euclidean distance calculation, Ward clustering and contour coefficient evaluation are carried out on the data, and dimensionality reduction and clustering are carried out on the multi-dimensional features. Therefore, comprehensive indexes with obvious differences are provided. According to the comprehensive indexes obtained through the method, original data can be reserved, the complexity of the data can be greatly reduced, the analysis process is simplified, and an intuitive and operable means is provided for overall state evaluation of the environmental microbial community, so that the analysis efficiency and accuracy are improved, and the method is suitable for popularization and application. And determining the overall influence of the environmental change on the microbial community state.

Description

Technical Field

[0001] The present invention belongs to the field of environmental microbial technology and ecological health technology, and specifically relates to an environmental microbial community status assessment method based on unsupervised clustering. Background Art

[0002] In recent years, with the rapid development of high-throughput sequencing technology, research on microbial communities has been widely used in the field of environmental science. The state of microbial communities is often described by a variety of indicators, including diversity (such as species richness and Shannon index), community composition (such as species abundance and distribution), ecological functional characteristics (such as element cycling), etc. These indicators reveal the ecological characteristics and functional status of environmental microbial communities from different perspectives, and can provide important information for ecological environmental monitoring and governance.

[0003] Then, the environmental microbial community is essentially a complex multidimensional system, and there may be correlations and influences between various indicators. In most current studies, these indicators are often calculated and analyzed independently, lacking comprehensive indicators that can fully integrate the multidimensional characteristics of environmental microorganisms. This not only increases the complexity of data analysis, but also limits the overall assessment and understanding of the state of environmental microbial communities. Therefore, it is particularly important to develop a technology that can effectively reduce the dimension and cluster multidimensional features and generate comprehensive indicators that can comprehensively reflect the state of environmental microbial communities.

[0004] Unsupervised clustering is a technique for classification and analysis based on the intrinsic structure of data. It can automatically identify patterns and group structures between samples without prior knowledge or labels. By applying unsupervised clustering methods to environmental microbial community data analysis, multi-dimensional features can be reduced and clustered to extract comprehensive indicators. This comprehensive indicator can not only retain the main information of the original data, but also greatly reduce the complexity of the data, simplify the analysis process, and provide an intuitive and operational means for the overall status assessment of environmental microbial communities. Summary of the invention

[0005] In view of the above problems, the object of the present invention is to provide a method for evaluating the status of environmental microbial communities based on unsupervised clustering.

[0006] The specific technical solutions are as follows:

[0007] A method for evaluating the status of environmental microbial communities based on unsupervised clustering, comprising the following steps: 1) sample collection and quality control: collecting metagenomic sequencing data of target environmental samples, performing quality control on the raw metagenomic sequencing data, removing low-quality sequences, and obtaining valid metagenomic sequencing data for subsequent analysis; 2) microbial species annotation and abundance calculation: performing species annotation on the valid metagenomic data after quality control in step 1), performing microbial species abundance calculation, and defining key species according to abundance, i.e., the top 5 species in terms of abundance;

[0008] 3) Microbial diversity calculation: Calculate the sample microbial diversity based on the microbial species abundance data, namely the species richness and the species Shannon index (Shannon diversity index);

[0009] 4) Microbial functional gene annotation and abundance calculation: Perform functional gene annotation on the effective metagenomic data after quality control in step 1) and calculate the abundance of microbial functional genes;

[0010] 5) Reduce the multidimensional characteristics of microorganisms to comprehensive indicators: standardize key species, species richness, Shannon index and abundance of microbial functional genes, use PCA to reduce the dimension of standardized data of microbial community characteristics, retain 95% of the principal components of variance for clustering, use Euclidean distance to calculate the distance between data points based on the reduced data, cluster the data using Ward hierarchical clustering algorithm, use silhouette coefficient to evaluate the clustering effect, and finally divide the data into comprehensive indicators.

[0011] Furthermore, in step 5), the microbial community characteristic data is standardized, and the standardized calculation formula is:

[0012]

[0013] Among them, X is the feature data, μ is the mean of the feature, and σ is the standard deviation of the feature. The principal component analysis (PCA) is used to reduce the dimension of the microbial community feature data, and the calculation method is:

[0014]

[0015] Where C is the covariance matrix, x i is the feature data, is the average value of the feature data, n is the number of samples, and the eigenvalues ​​and eigenvectors of the covariance matrix are calculated to obtain the new principal component directions.

[0016] The principal component with 95% variance is retained for clustering, and the variance rate is calculated as:

[0017]

[0018] Among them, R represents the variance explanation rate, λ i is the eigenvalue of the i-th principal component, and m is the number of eigenvectors.

[0019] Furthermore, in step 5), the Euclidean distance is used to calculate the distance between data points for the environmental microbial community data after dimensionality reduction. The Euclidean distance calculation formula is:

[0020]

[0021] Among them, d represents the Euclidean distance, x and y represent two sample points respectively, x i represents the i-th feature value of sample point x, y i Represents the i-th feature value of sample point y.

[0022] Furthermore, in step 5), after calculating the distance between samples, the Ward hierarchical clustering algorithm is used to cluster the data, and how to merge clusters is determined by minimizing the variance within the cluster. The formula of the Ward hierarchical clustering algorithm is:

[0023]

[0024] in, and are the centers of clusters A and B (the mean of samples within the cluster), |A| and |B| are the sizes of the clusters (the number of samples within the cluster), represents the center distance between two clusters, ΔE represents the cost function after cluster merging, and represents the change in variance within the cluster.

[0025] Furthermore, after clustering in step 5), the silhouette coefficient is used to evaluate the clustering effect. It measures the difference between the similarity of each sample to its cluster and its similarity to the nearest cluster. The calculation formula of the silhouette coefficient is:

[0026]

[0027] Among them, a(i) is the average distance from sample i to other samples in its cluster, b(i) is the average distance from sample i to the nearest cluster, and the value range of the silhouette coefficient is [-1,1]. The closer it is to 1, the better the clustering effect.

[0028] The beneficial effects of the present invention are:

[0029] This method analyzes the intrinsic structure of the data through unsupervised clustering methods, identifying patterns and group structures between samples without prior knowledge. In the analysis of environmental microbial community data, this method can reduce the dimension and cluster multidimensional features, extract comprehensive indicators, simplify analysis, reduce complexity, and retain key information. This method provides an intuitive and operational means for the assessment of the status of environmental microbial communities, improves analysis efficiency and accuracy, helps to assess the impact of environmental changes on microbial communities, and provides a reference for the construction of ecological civilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of the process of the present invention DETAILED DESCRIPTION

[0031] The present invention is further analyzed below in conjunction with specific embodiments, but the protection scope of the present invention is not limited thereto.

[0032] Example 1, unsupervised clustering assessment of the state of environmental microbial communities in the ocean, the detailed process is as follows Figure 1 shown.

[0033] Specific steps:

[0034] 1. Collection of sampling information of marine metagenome data: Sampling seawater samples at different seawater depths and different latitudes to obtain raw metagenome sequencing data. Quality control of raw marine metagenome sequencing data: First, quality control and preprocessing of raw marine metagenome sequencing data are required to remove low-quality sequences and adapters, and obtain valid marine metagenome sequencing data for subsequent analysis.

[0035] 2. Marine microbial species annotation and abundance calculation: perform species annotation and abundance calculation on the effective metagenomic data after quality control. Define key species based on abundance, i.e. the top 5 species in abundance ranking.

[0036] 3. Calculation of marine microbial species diversity: Calculate the microbial diversity (species richness and Shannno index) in the sample based on the marine microbial species abundance data.

[0037] 4. Marine microbial functional gene annotation and abundance calculation: Perform functional gene annotation and abundance calculation on the effective metagenome data. In this example, the functional genes selected are key genes involved in marine carbon, nitrogen, phosphorus and sulfur.

[0038] 5. Based on the key species, species richness, Shannon index and abundance of key genes of marine carbon, nitrogen, phosphorus and sulfur obtained in steps 3, 4 and 5, standardization is performed. The standardization formula is:

[0039]

[0040] Among them, X is the feature data, μ is the mean of the feature, and σ is the standard deviation of the feature.

[0041] 6. Based on the standardized data of microbial community characteristics obtained in step 5, the dimension of the microbial community characteristic data is reduced by principal component analysis, and the calculation method is:

[0042]

[0043] Where C is the covariance matrix, x i is the sample point, is the average value of the sample, n is the number of samples, calculate the eigenvalues ​​and eigenvectors of the covariance matrix, and get the new principal component direction. The variance rate calculation formula is:

[0044]

[0045] Among them, R represents the variance explanation rate, λ i is the eigenvalue of the i-th principal component, and m is the number of eigenvectors.

[0046] 7. Based on the environmental microbial community data after dimension reduction in step 6, use the Euclidean distance to calculate the distance between data points. The Euclidean distance calculation formula is:

[0047]

[0048] Among them, d represents the Euclidean distance, x and y represent two sample points respectively, x i represents the i-th feature value of sample point x, y i Represents the i-th feature value of sample point y.

[0049] 8. Based on step 7, after calculating the distance between samples, use the Ward hierarchical clustering algorithm to cluster the data and decide how to merge clusters by minimizing the variance within the cluster. The formula of the Ward hierarchical clustering algorithm is:

[0050]

[0051] in, and are the centers of clusters A and B (the means of the samples within the clusters), |A| and |B| are the sizes of the clusters (the number of samples within the clusters), represents the center distance between two clusters, ΔE represents the cost function after cluster merging, and represents the change in variance within the cluster. After merging, there are seven comprehensive indicators, generating seven ecological states. These ecological states show significant differences in the characteristics of the microbial community.

[0052] 9. After clustering is completed based on step 8, the silhouette coefficient is used to evaluate the clustering effect. It measures the difference between the similarity of each sample to its cluster and its similarity to the nearest cluster. The silhouette coefficient calculation formula is:

[0053]

[0054] Among them, a(i) is the average distance from sample i to other samples in its cluster, b(i) is the average distance from sample i to the nearest cluster, and the average value of the silhouette coefficient is 0.91, indicating that the clustering effect is good.

[0055] Example conclusion:

[0056] The present invention provides a method for evaluating the state of environmental microbial communities based on unsupervised clustering. For marine microbial communities, the hierarchical clustering using the ward clustering algorithm based on Euclidean distance has the highest silhouette coefficient, indicating that the clustering effect of this hyperparameter combination is the best. The 7 comprehensive indicators established based on this hyperparameter and algorithm combination divide the marine microbial community into 7 major types, with an average silhouette coefficient of 0.91. This method can not only effectively save experimental manpower and material costs, but also has high reliability. Compared with traditional single environment experiments, the present invention overcomes the influence of environment, sequencing technology and sampling difficulties, and provides a more comprehensive and reliable approach for evaluating the state of marine microbial communities.

Claims

1. A method for evaluating the status of environmental microbial communities based on unsupervised clustering, characterized in that: The steps include: 1) Sample collection and quality control: Collect metagenomic sequencing data of target environmental samples, perform quality control on the raw data of metagenomic sequencing, remove low-quality sequences, and obtain valid metagenomic sequencing data for subsequent analysis; 2) Microbial species annotation and abundance calculation: perform species annotation on the effective metagenomic data after quality control in step 1), calculate the abundance of microbial species, and define key species based on abundance, i.e., the top 5 species in terms of abundance; 3) Microbial diversity calculation: Calculate sample microbial diversity based on microbial species abundance data, namely species richness and Shannon index; 4) Microbial functional gene annotation and abundance calculation: perform microbial functional gene annotation on the effective metagenomic data after quality control in step 1), and then calculate the abundance of microbial functional genes; 5) Reduce the multidimensional characteristics of microorganisms to comprehensive indicators: standardize key species, species richness, Shannon index and abundance of microbial functional genes, use principal component analysis (PCA) to reduce the dimension of standardized data of microbial community characteristics, retain 95% of the principal components for clustering, use Euclidean distance to calculate the distance between data points based on the reduced data, cluster the data using Ward hierarchical clustering algorithm, use silhouette coefficient to evaluate the clustering effect, and finally divide the data into comprehensive indicators.

2. The method for evaluating the state of environmental microbial communities based on unsupervised clustering according to claim 1, characterized in that: In step 5), the microbial community characteristic data is standardized, and the standardized calculation formula is: Among them, X is the feature data, μ is the mean of the feature, and σ is the standard deviation of the feature; PCA was used to reduce the dimension of the standardized data of microbial community characteristics, and the calculation formula was: Where C is the covariance matrix, x i is the sample point, is the mean value of the sample, n is the number of samples, calculate the eigenvalues ​​and eigenvectors of the covariance matrix, and get the new principal component direction. The principal component with 95% variance is retained for clustering, and the variance rate is calculated as: Among them, R represents the variance explanation rate, λ i is the eigenvalue of the i-th principal component, and m is the number of eigenvectors.

3. The method for evaluating the state of environmental microbial communities based on unsupervised clustering according to claim 2, characterized in that: In step 5), the Euclidean distance is used to calculate the distance between data points for the environmental microbial community data after dimensionality reduction. The Euclidean distance calculation formula is: Among them, d represents the Euclidean distance, x and y represent two sample points respectively, x i represents the i-th feature value of sample point x, y i Represents the i-th feature value of sample point y.

4. The method for evaluating the state of environmental microbial communities based on unsupervised clustering according to claim 3, characterized in that: In step 5), after calculating the distance between samples, the Ward hierarchical clustering algorithm is used to cluster the data. The Ward hierarchical clustering algorithm formula is as follows: in, and are the centers of clusters A and B respectively (the center of a cluster can be the mean of all samples in the cluster), |A| and |B| are the sizes of the clusters (the number of samples in the cluster), represents the center distance between two clusters, ΔE represents the cost function after cluster merging, and represents the change in variance within the cluster.

5. The method for evaluating the state of environmental microbial communities based on unsupervised clustering according to claim 4, characterized in that: After the clustering in step 5) is completed, the silhouette coefficient is used to evaluate the clustering effect. It measures the difference between the similarity of each sample to its cluster and its similarity to the nearest cluster. The silhouette coefficient calculation formula is: Among them, a(i) is the average distance from sample i to other samples in its cluster, b(i) is the average distance from sample i to the nearest cluster, and the value range of the silhouette coefficient is [-1,1]. The closer it is to 1, the better the clustering effect.

Images