A method for predicting properties of dissolved organic matter in lakes based on machine learning

Abstract

The present invention discloses a method for predicting the properties of lake dissolved organic matter based on machine learning, by collecting lake water samples; by detecting the physical and chemical properties, microbial components, and dissolved organic matter components of the water samples, collecting humanistic indicators and land use type data of the lake and its basin, and establishing a lake database; using a machine learning feature selection method to screen the lake feature set to obtain a screened feature set; based on a machine learning algorithm, the screened data set is input into a machine learning random forest model for training, and the parameters of the machine learning random forest are adjusted to meet the accuracy requirements, to obtain a machine learning model for predicting the properties of lake dissolved organic matter; according to the adjusted random forest model, 10-fold cross validation is performed to obtain the prediction results of the model. The present invention quickly and accurately predicts the properties of lake dissolved organic matter, and provides a technical basis for quickly judging the properties of lake dissolved organic matter.

Description

Technical Field

[0001] The present invention relates to the field of environmental technology, and in particular to a method for predicting properties of dissolved organic matter in lakes based on machine learning. Background Art

[0002] The intensification of global carbon emissions has led to global warming. Inland lakes are considered to be an important source of carbon emissions. Understanding the composition and properties of dissolved organic matter in lakes helps to understand the carbon cycle trend of lakes. The detection of dissolved organic matter components is mainly carried out through Fourier transform ion cyclotron resonance mass spectrometry, but due to its high cost and complex analysis steps, it consumes a lot of manpower and material resources. With the emergence of methods such as machine learning, it can handle the relationship between multiple variables and labels, and make fast and accurate predictions at the same time. By screening the data set and adjusting the model parameters, it can achieve better prediction results, but it is less used in dissolved organic matter in lakes. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art and fill the gaps, and to provide a method for predicting the properties of dissolved organic matter in lakes based on machine learning, which can achieve accurate and rapid prediction of the properties of dissolved organic matter in lakes.

[0004] To achieve the above objectives, the present invention provides a solution to the above technical problems as follows: a method for predicting the properties of dissolved organic matter in lakes based on machine learning, comprising the following steps:

[0005] Step S1 collects water samples from 40 lakes in China;

[0006] Step S2 detects the physical and chemical properties, microbial components, and dissolved organic matter components of the water sample, collects human indicators and land use type data of the lake and its basin, and establishes a lake database;

[0007] Step S3 uses a machine learning feature selection method to perform feature screening on the feature data set to obtain a screening feature set;

[0008] Step S4: inputting the screening data set into the machine learning random forest model for training based on the machine learning algorithm, and adjusting the parameters of the machine learning random forest to meet the accuracy requirements, thereby obtaining a machine learning model for predicting the properties of dissolved organic matter in lakes;

[0009] Step S5 performs 10-fold cross validation based on the adjusted random forest model to obtain the prediction result of the model.

[0010] Furthermore, in step S1, water samples from 40 lakes in China are collected, and the specific steps include:

[0011] Step S1.1 collects water samples from 40 lakes in China;

[0012] Furthermore, in step S2, the physical and chemical properties, microbial components, and dissolved organic matter components of the water sample are detected, human indicators and land use type data of the lake and its basin are collected, and a lake database is established. The specific steps include:

[0013] Step S2.1: Detect the total nitrogen content, chemical oxygen demand, microbial composition (abundance of fungal and bacterial phyla, Chao and Shannon indexes), and dissolved organic matter composition of the water sample;

[0014] Step S2.2: The latitude and longitude, altitude, average annual rainfall, average annual temperature, air quality index, GDP density, population density, and land use type (artificial surface, agricultural land, water body, wetland, forest, shrub grassland, and others) of the lake basin were collected through maps, databases, government gazettes, and yearbooks to establish a lake database;

[0015] Step S2.3 calculates two properties of the measured dissolved organic matter components: degradation index and bioavailability. The formula of the degradation index is:

[0016] I deg =∑(I deg,neg ) / ∑(I deg,neg +I deg,pos )

[0017] Where: I neg,neg : Negatively correlated with dissolved organic matter abundance, C 21 H 26 O 11 ,C 17 H 20 O9,C 19 H 22 O 10 ,C 20 H 22 O 10 ,C 20 H 24 O 11 ;I deg,pos : Positively correlated with dissolved organic matter abundance, C 13 H 18 O7,C 14 H 20 O7,C 15 H 22 O7,C 15 H 22 O8 16 H 24 O8;

[0018] The bioavailability formula is:

[0019] Biodegradability = 38.4 + 10.6 (J:C) - 70.9 (O:C) + 183.2 (N:C)

[0020] Where: H, C, O, N represent the number of hydrogen atoms, carbon atoms, oxygen atoms and nitrogen atoms in the chemical formula respectively.

[0021] Furthermore, in step S3, the feature data set is subjected to feature screening using a machine learning feature selection method to obtain a screening feature set, and the specific steps include:

[0022] Step S3.1 uses an iterative method based on the random forest model to eliminate features with negative importance to screen features until no negative feature importance appears, and selects the feature set with the best model effect.

[0023] Step S3.2: Through feature screening, the features of the degradation index include: annual average temperature, artificial surface, water body, population density, air quality, abundance of bacteria Latescibacterota, abundance of bacteria Margulisbacteria, and abundance of bacteria Methylomirabilota;

[0024] Step S3.3: Through feature screening, the features of bioavailability include: longitude, altitude, forest, shrub grassland, artificial surface, GDP density, population density, fungal Chao index, bacterial Cyanobacteria abundance, bacterial Chytridiomycota abundance, and bacterial Rozellomycota abundance.

[0025] Furthermore, in step S4, based on the machine learning algorithm, the screening data set is input into the machine learning random forest model for training, and the parameters of the machine learning random forest are adjusted to meet the accuracy requirements, so as to obtain a machine learning model for predicting the properties of dissolved organic matter in lakes. The specific steps include:

[0026] Step S4.1: Input the screened data set into the machine learning random forest model and use the ten-fold cross validation method to perform model validation;

[0027] The data set was randomly divided into 10 sub-datasets in a ratio of 9:1. Nine different sub-datasets were selected and merged into the training set for input into the model for training. The remaining 1 subset was used as the test set to obtain the test results. The model prediction accuracy evaluation index is: correlation coefficient (R 2 );

[0028] R 2 =ρ X,Y

[0029] Among them, X represents the actual value and Y represents the predicted value;

[0030] Step S4.2 adjusts the parameters of the random forest model according to the model prediction accuracy verification results. The adjusted parameters include: the number of trees ntree and the number of optimal node variables mtry.

[0031] Specific model expression:

[0032] Build models based on R language;

[0033] Import the randomForest and Metrics packages;

[0034] Read the filtered data set;

[0035] Set random number;

[0036] The random forest model was established using the randomForest function, where the parameters were set as: ntree=600, mtry=2.

[0037] The beneficial effects of the present invention are as follows

[0038] 1. The present invention provides a method for predicting the properties of dissolved organic matter in lakes based on machine learning. The method uses the physical and chemical properties of lake water samples, microbial components, and human indicators and land use type data collected from lakes and their basins as feature input models. The degradation index and bioavailability are calculated through the dissolved organic matter components as label outputs. Through machine learning random forest model training and learning, a stable R model that can be used to predict the properties of dissolved organic matter in lakes, the degradation index and bioavailability model are obtained. 2 As high as 0.74 and 0.86 respectively.

[0039] 2. The method for predicting the properties of dissolved organic matter in lakes based on machine learning of the present invention can quickly and accurately predict the properties of dissolved organic matter in lakes, providing a technical basis for quickly predicting the properties of dissolved organic matter in lakes. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of the method for predicting properties of lake dissolved organic matter based on machine learning of the present invention;

[0041] Figure 2 A flow chart of the method for machine learning feature screening of the present invention;

[0042] Figure 3 The main algorithm flow chart of the embodiment method of the present invention;

[0043] Figure 4 This is a graph of the ten-fold cross-validation results for predicting the properties of dissolved organic matter in lakes in the specific implementation method of the present invention. DETAILED DESCRIPTION

[0044] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0045] The present invention provides a method for predicting the properties of lake dissolved organic matter based on machine learning, which can quickly and accurately obtain the results of the properties of lake dissolved organic matter based on the data set of the lake and its basin.

[0046] like Figure 1 As shown, the present invention comprises the following steps:

[0047] S1 collected water samples from more than 40 lakes in China;

[0048] S2 tests the physical and chemical properties, microbial components, and dissolved organic matter components of water samples, collects data on human indicators and land use types of lakes and their watersheds, and establishes a lake database;

[0049] S3 uses a machine learning feature selection method to perform feature screening on the feature data set to obtain a screened feature set;

[0050] S4 inputs the screened data set into the machine learning random forest model for training based on the machine learning algorithm, and adjusts the parameters of the machine learning random forest to meet the accuracy requirements, thereby obtaining a machine learning model for predicting the properties of dissolved organic matter in lakes;

[0051] S5 performs 10-fold cross validation based on the adjusted random forest model to obtain the prediction results of the model.

[0052] Furthermore, the step S1 collects water samples from 40 lakes in China, and the specific steps include:

[0053] S1.1 Collect water samples from 40 lakes in China;

[0054] Furthermore, the step S2 detects the physical and chemical properties, microbial components, and dissolved organic matter components of the water sample, collects humanistic indicators and land use type data of the lake and its basin, and establishes a lake database. The specific steps include:

[0055] S2.1 Test water samples for: total nitrogen content, chemical oxygen demand, microbial composition (abundance of fungal and bacterial phyla, Chao and Shannon indexes), and dissolved organic matter composition;

[0056] S2.2 The latitude and longitude, altitude, average annual rainfall, average annual temperature, air quality index, GDP density, population density, and land use type (artificial surface, agricultural land, water body, wetland, forest, shrub grassland, and others) of lake basins were collected through maps, databases, government gazettes, and yearbooks, and a lake database was established;

[0057] S2.3 Calculate two properties of the measured dissolved organic matter components: degradation index and bioavailability. The formula for the degradation index is:

[0058] I deg =∑(I deg,neg ) / ∑(I deg,neg +I deg,pos )

[0059] Where: I deg,neg : Negatively correlated with dissolved organic matter abundance, C 21 H 26 O 11 ,C 17 H 20 O9,C 19 H 22 O 10 ,C 20 H 22 O 10 ,C 20 H 24 O 11 ;I deg,pos : Positively correlated with dissolved organic matter abundance, C 13 H 18 O7,C 14 H 20 O7,C 15 H 22 O7,C 15 H 22 O8 16 H 24 O8;

[0060] The bioavailability formula is:

[0061] Biodegradability = 38.4 + 10.6 (H:C) - 70.9 (O:C) + 183.2 (N:C)

[0062] Where: H, C, O, N represent the number of hydrogen atoms, carbon atoms, oxygen atoms and nitrogen atoms in the chemical formula respectively.

[0063] Furthermore, in step S3, feature selection is performed on the feature data set using a machine learning feature selection method, and the specific steps include:

[0064] S3.1 uses an iterative method based on the random forest model to remove features with negative importance to screen features until no negative feature importance appears, and selects the feature set with the best model effect, such as Figure 2 As shown, where: X: feature space, d: feature space dimension, x -:The features with negative MSE increase values. Then the model is evaluated using the ten-fold cross validation method to select the feature set with the best model effect.

[0065] S3.2 Through feature screening, the characteristics of the degradation index include: annual average temperature, artificial surface, water body, population density, air quality, abundance of bacteria Latescibacterota, abundance of bacteria Margulisbacteria, and abundance of bacteria Methylomirabilota;

[0066] S3.3 Through feature screening, the characteristics of bioavailability include: longitude, altitude, forest, shrub grassland, artificial surface, GDP density, population density, fungal Chao index, bacterial Cyanobacteria abundance, bacterial Chytridiomycota abundance, and bacterial Rozellomycota abundance.

[0067] Furthermore, the step S4 inputs the screening data set into the machine learning random forest model for training based on the machine learning algorithm, and adjusts the parameters of the machine learning random forest to meet the accuracy requirements, thereby obtaining a machine learning model for predicting the properties of dissolved organic matter in lakes. The specific steps include:

[0068] S4.1 Input the screened data set into the machine learning random forest model and use the ten-fold cross validation method to perform model validation;

[0069] The data set was randomly divided into 10 sub-datasets in a ratio of 9:1. Nine different sub-datasets were selected and merged into the training set for input into the model for training. The remaining 1 subset was used as the test set to obtain the test results. The model prediction accuracy evaluation index is: correlation coefficient (R 2 );

[0070] R 2 =ρ X,Y

[0071] Among them, X represents the actual value and Y represents the predicted value;

[0072] S4.2 The parameters of the random forest model are adjusted according to the model prediction accuracy verification results. The adjusted parameters include: the number of trees ntree and the number of optimal node variables mtry.

[0073] Figure 3 The main algorithm flow chart of the embodiment method of the present invention;

[0074] Figure 4 As shown, the average degradation index R of the ten-fold cross validation of the present invention is 2 for:

[0075] (0.78+0.87+0.65+0.47+0.69+0.28+0.83+0.96+0.95+0.89) / 10=0.74

[0076] The average bioavailability R 2 for:

[0077] (0.88+0.84+0.80+1.00+0.83+0.88+0.57+0.89+0.92+0.97) / 10=0.86

[0078] In summary, the present invention provides a method for predicting the properties of dissolved organic matter in lakes based on machine learning. Using a dataset of 40 lakes in China as input data, a stable and accurate random forest model for predicting the properties of dissolved organic matter in lakes is obtained through training and learning of a machine learning random forest model. The R2 of the degradation index is as high as 0.74, and the R2 of the bioavailability is as high as 0.86. The method for predicting the properties of dissolved organic matter in lakes based on a machine learning algorithm of the present invention can quickly and accurately predict the properties of dissolved organic matter in lakes, providing a technical basis for quickly predicting the properties of dissolved organic matter in lakes.

[0079] The present invention is not limited to the above-mentioned example method, and the protection scope of the present invention cannot be limited thereto. Any changes made on the basis of the technical solution in accordance with the technical ideas proposed by the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for predicting the properties of dissolved organic matter in lakes based on machine learning, It is characterized in that The steps include: S1 collected water samples from more than 40 lakes in China; S2 tests the physical and chemical properties, microbial components, and dissolved organic matter components of water samples, collects data on human indicators and land use types of lakes and their watersheds, and establishes a lake database; S3 uses a machine learning feature selection method to perform feature screening on the feature data set to obtain a screened feature set; S4 inputs the screened data set into the machine learning random forest model for training based on the machine learning algorithm, and adjusts the parameters of the machine learning random forest to meet the accuracy requirements, thereby obtaining a machine learning model for predicting the properties of dissolved organic matter in lakes; S5 performs ten-fold cross validation based on the adjusted random forest model to obtain the prediction results of the model; In step S2, the physical and chemical properties, microbial components, and dissolved organic matter components of the water sample are detected, human indicators and land use type data of the lake and its basin are collected, and a lake database is established. The specific steps include: S2.1 Test water samples for: total nitrogen content, chemical oxygen demand, microbial composition including fungal and bacterial abundance, Chao and Shannon index, and dissolved organic matter composition; S2.2 Collect the latitude and longitude, altitude, average annual rainfall, average annual temperature, air quality index, GDP density, population density, land use types including artificial surface, agricultural land, water bodies, wetlands, forests, shrub grasslands and others of lake basins through maps, databases, government gazettes and yearbooks, and establish a lake database; S2.3 Calculate two properties of the measured dissolved organic matter components: degradation index and bioavailability. The formula for the degradation index is: I deg =∑(I deg,neg ) / ∑(I deg,neg +I deg,pos ) Where: I deg,neg : Negatively correlated with dissolved organic matter abundance, C 21 H 26 O 11 ,C 17 H 20 O 9 ,C 19 H 22 O 10 ,C 20 H 22 O 10 ,C 20 H 24 O 11 ;I deg,pos : Positively correlated with dissolved organic matter abundance, C 13 H 18 O 7 ,C 14 H 20 O 7 ,C 15 H 22 O 7 ,C 15 H 22 O 8 andC 16 H 24 O 8 ; The bioavailability formula is: Biodegradability = 38.4 + 10.6 (H:C) - 70.9 (O:C) + 183.2 (N:C) Where: H, C, O, N represent the number of hydrogen atoms, carbon atoms, oxygen atoms and nitrogen atoms in the chemical formula respectively; The step S4 inputs the screening data set into the machine learning random forest model for training based on the machine learning algorithm, and adjusts the parameters of the machine learning random forest to meet the accuracy requirements, so as to obtain a machine learning model for predicting the properties of dissolved organic matter in lakes. The specific steps include: S4.1 Input the screened data set into the machine learning random forest model and use the ten-fold cross validation method to perform model validation; The data set was randomly divided into 10 sub-datasets in a ratio of 9:

1. Nine different sub-datasets were selected and merged into the training set for input into the model for training. The remaining 1 subset was used as the test set to obtain the test results. The model prediction accuracy evaluation index is: correlation coefficient (R 2 ); R 2 =ρ X,Y Among them, X represents the actual value and Y represents the predicted value; S4.2 The parameters of the random forest model are adjusted according to the model prediction accuracy verification results. The adjusted parameters include: the number of trees ntree and the number of optimal node variables mtry.

2. The method for predicting the properties of dissolved organic matter in lakes based on machine learning according to claim 1, It is characterized in that In step S3, feature selection is performed on the feature data set using a machine learning feature selection method, and the specific steps include: S3.1 uses an iterative method based on the random forest model to remove features with negative importance to screen features until no negative feature importance appears, and selects the feature set with the best model effect; S3.2 Through feature screening, the characteristics of the degradation index include: annual average temperature, artificial surface, water body, population density, air quality, abundance of bacteria Latescibacterota, abundance of bacteria Margulisbacteria, and abundance of bacteria Methylomirabilota; S3.3 Through feature screening, the characteristics of bioavailability include: longitude, altitude, forest, shrub grassland, artificial surface, GDP density, population density, fungal Chao index, bacterial Cyanobacteria abundance, bacterial Chytridiomycota abundance, and bacterial Rozellomycota abundance.

Images