Fly maggot environment adaptability prediction method
Through the combination of regression algorithm and generalized additive model, important features in the adaptive prediction of magpie environment were screened out, variable selection and data quality problems were solved, the accuracy and interpretability of the model were improved, and better prediction results were achieved .
Patent Information
- Application Number
- CN202410886958.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-03
- Publication Date
- 2024-11-22
AI Technical Summary
In biological environmental aligning prediction, excessive selection of variables leads to poor performance of the model, making it difficult to explain the impact of key environmental variables on species aligning, and data quality issues affect the prediction results.
Regression algorithm is used to filter environmental variables, build and train with generalized additive models, combine cross-validation and regularization terms to determine important features, reduce the number of input variables, reduce the risk of overfitting, and predict through the autoregressive integral moving average model. .
The accuracy, interpretability, robustness and computational efficiency of the model are improved, and important features are automatically selected, which reduces the complexity of the model, making the prediction results closer to the real value.
Smart Images

Figure CN119004248A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of biological environment adaptability prediction, in particular to a method for predicting the environmental adaptability of fly maggots. Background Art
[0002] The biological environment suitability prediction technology uses multidisciplinary methods and tools such as statistical analysis, ecology, geographic information systems, and machine learning to evaluate and predict the adaptability and survival possibility of specific organisms in a specific environment, providing a scientific basis for ecological protection, environmental management, and resource utilization.
[0003] Regression is a statistical method and machine learning technique used to analyze the relationship between variables and then predict the output variable from the input variable. Regression algorithm builds a mathematical model to describe the relationship between the input variable and the output variable, so as to achieve the goal of predicting new data.
[0004] In the process of predicting biological environmental suitability, variable selection is a crucial step; in the process of variable selection, the selection of too many features will lead to poor performance of the model on new data, and it will be difficult to explain which environmental variables have a key impact on species suitability; variable selection also has collinearity problems, and there is a high correlation between variables, which leads to model instability and difficulty in clarifying the independent contribution of each variable to species suitability; in addition, data quality issues such as missing values, outliers and errors may affect the prediction results. Summary of the invention
[0005] The purpose of the present invention is to provide a method for predicting the environmental suitability of fly maggots to solve the problems raised in the above background technology.
[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for predicting the environmental suitability of fly maggots, the method comprising the following steps: S1. Collect and organize environmental variable data and fly maggot quantity data under corresponding variables; S2, integrating the data collected in step S1 into a data set and dividing it into a training set and a test set; S3. Use The regression algorithm screens environmental variables; S4. Building a generalized additive model , train and evaluate the model; S5. Use the trained model to make predictions for known environmental variables or areas to be tested.
[0007] In step S1, the environmental variable data refers to the state of environmental factors that affect the survival of fly maggots; the fly maggot quantity data refers to the number of fly maggots per unit area; wherein the environmental factor is a characteristic variable, which can be expressed as , is the number of features; the number of fly maggots is the target variable, which can be expressed as ; Feature variables are used to predict target variables.
[0008] In step S2, a cross-validation method is used to divide the training set and the test set; Divide the data set into K parts in equal proportions, and conduct K experiments in total. In the first experiment, select the first part of the data as test data, and the remaining K-1 parts of the data as training data. In the second experiment, select the second part of the data as test data, and the remaining K-1 parts of the data as training data. In the Kth experiment, select the Kth part of the data as test data, and the remaining K-1 parts of the data as training data. In each experiment, you will get a The mean square error value is calculated by averaging the K mean square error values to get the cross validation error.
[0009] The mean square error value can be calculated by the formula:
[0010] in, represents the number of test set samples, represents the true value, Represents the predicted value; in regression analysis, the mean square error is used to measure the average square error between the predicted value and the actual value; from the formula, it can be seen that the mean square error is always greater than or equal to zero; when the predicted value is the same as the true value, the mean square error is 0, so the closer the mean square error is to 0, the closer the predicted value is to the true value; similarly, the smaller the cross-validation error is, the better the model performance is.
[0011] In step S3, use Regression algorithms identify important environmental variables; The regression algorithm adds restrictions on weights as a regularization term, placing limited weights on more important feature dimensions:
[0012] in, is the number of samples, represents the true value, represents the predicted value, is the regularization parameter, for norm; Regularization parameter Controls the strength of regularization, The larger the value, the greater the penalty for model complexity, and more environmental variable coefficients are compressed to 0; The smaller the value, the closer it is to the general linear regression; retain the environmental variables whose coefficients are not 0; as The value of the environmental variable increases, and the later the environmental variable coefficient becomes 0, the more important the variable is in the model; using the iterative fitting along the regularization path Linear Model , by specifying the length of the regularization path, that is, limiting The ratio of minimum to maximum values is automatically generated Or manually set the regularization parameter The value of controls the penalty on model complexity to change the number of important features; The model regression coefficient vector can be expressed as ,in is the number of features; then The norm can be calculated as ; That is, sum up the absolute values of all regression coefficients; Adjusted regression coefficients make The value of is minimized to achieve the purpose of controlling the complexity of the model through regularization while considering the accuracy; the regression coefficient is adjusted by using the coordinate descent method make In the process of minimizing the value, the features whose regression coefficients are not 0 are important features.
[0013] In step S4, the generalized additive model is used To analyze and predict the relationship between variables, the model formula is usually expressed as:
[0014] in, represents the dependent variable, which is the target variable to be predicted; is the intercept term, which means that when all characteristic variables The expected target variable value when all values are 0; , , , is a nonlinear function used to represent the nonlinear relationship between the feature vector and the target vector; represents the error term, which is the random part that the model cannot explain; Fitting using training set data Model; mean squared error value obtained using the test set Evaluate the model.
[0015] In step S5, the known environmental variables represent the collected Characteristic variables, the variables collected during the collection process do not include the target variable, that is, the number of species; after step S1 S4 achieves the purpose of predicting the target variable by using the characteristic variables.
[0016] In step S5, the prediction of the area to be measured includes the following preparations: S8-1. Collect historical environmental variables in the area; S8-2, predict the future environmental variables of the region; S8-3, select the environmental characteristic variables at the required time nodes to predict the suitability of the area; In step S8-1, by The climate database obtains the historical environmental variable data of the region, and the acquired features are the same as the features collected in step S1 Consistent.
[0017] In step S8-2, the autoregressive integrated moving average model is used to For fitting, use the trained The model predicts the future environmental variables in the region by: cleaning and standardizing time series data, drawing time series graphs, making non-stationary data stationary through differences, and using autocorrelation functions. and partial autocorrelation function Figure determination and , use the number of differences in data to determine , use the determined parameters to fit Model and use of trained The model makes predictions about future values; Among them, the autoregressive order Determined Autoregression Partially determines the current value and the previous The relationship between the hysteresis values; by the number of differences Determined points The moving average order determines how many times the data should be differentiated to make it stationary. Determined Moving Average Partially determines the current value and the previous The relationship between the error terms; autoregression ,integral and moving average The three parts clearly The structure of the model; After the prediction is completed, the environmental characteristic variables of the required time nodes are selected and the results are obtained through step S1. S4 achieves the purpose of predicting the target variable by using the characteristic variables.
[0018] Compared with the prior art, the present invention has the following beneficial effects: The regression algorithm can automatically select important features and shrink the coefficients of unimportant features to 0, which can effectively screen out the features that have the greatest impact on the target variable. Feature selection reduces the number of input variables, reduces the risk of overfitting, and improves the generalization ability of the model. The regression algorithm selects fewer and more important features, making the model easier to understand and interpret. Generalized Additive Model It has good interpretability and captures the relationship between each feature and the target variable through nonlinear functions, making the interpretation more intuitive. Combining the advantages of the two methods can effectively improve the accuracy, interpretability, robustness and computational efficiency of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 The present invention is a schematic diagram of the steps of a method for predicting the environmental adaptability of fly maggots. DETAILED DESCRIPTION
[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0021] See also Figure 1 The present invention provides a technical solution: a method for predicting the environmental adaptability of fly maggots, the method comprising the following steps: S1. Collect and organize environmental variable data and fly maggot quantity data under corresponding variables; S2, integrating the data collected in step S1 into a data set and dividing it into a training set and a test set; S3. Use Regression algorithm is used to screen environmental variables; S4. Building a generalized additive model , train and evaluate the model; S5. Use the trained model to make predictions for known environmental variables or areas to be tested.
[0022] In step S1, the environmental variable data refers to the state of environmental factors that affect the survival of fly maggots; the fly maggot quantity data refers to the number of fly maggots per unit area; wherein the environmental factor is a characteristic variable, which can be expressed as , is the number of features; the number of fly maggots is the target variable, which can be expressed as ; Feature variables are used to predict target variables.
[0023] In step S2, a cross-validation method is used to divide the training set and the test set; Divide the data set into K parts in equal proportions, and conduct K experiments in total; in the first experiment, select the first part of the data as the test data, and the remaining K-1 parts of the data as the training data; in the second experiment, select the second part of the data as the test data, and the remaining K-1 parts of the data as the training data… In the Kth experiment, select the Kth part of the data as the test data, and the remaining K-1 parts of the data as the training data; in each experiment, an MSE mean square error value will be obtained, and the cross-validation error will be obtained by averaging the K mean square error values.
[0024] The mean square error value can be calculated by the formula:
[0025] in, represents the number of test set samples, represents the true value, Represents the predicted value; in regression analysis, the mean square error is used to measure the average square error between the predicted value and the actual value; from the formula, it can be seen that the mean square error is always greater than or equal to zero; when the predicted value is the same as the true value, the mean square error is 0, so the closer the mean square error is to 0, the closer the predicted value is to the true value; similarly, the smaller the cross-validation error is, the better the model performance is.
[0026] In step S3, use regression algorithms to identify important environmental variables; The regression algorithm adds restrictions on weights as a regularization term, placing limited weights on more important feature dimensions:
[0027] in, is the number of samples, represents the true value, represents the predicted value, is the regularization parameter, for norm; Regularization parameter Controls the strength of regularization, The larger the value, the greater the penalty for model complexity, and more environmental variable coefficients are compressed to 0; The smaller the value, the closer it is to the general linear regression; retain the environmental variables whose coefficients are not 0; as The value of the environmental variable increases, and the later the environmental variable coefficient becomes 0, the more important the variable is in the model; using the iterative fitting along the regularization path Linear Model , by specifying the length of the regularization path, that is, limiting The ratio of the minimum value to the maximum value is automatically generated Or manually set the regularization parameter The value of controls the penalty on model complexity to change the number of important features; The model regression coefficient vector can be expressed as ,in is the number of features; then The norm can be calculated as ; That is, sum up the absolute values of all regression coefficients; Adjusted regression coefficients Envoy The value of is minimized to achieve the purpose of controlling the complexity of the model through regularization while considering the accuracy; the regression coefficient is adjusted by using the coordinate descent method Envoy In the process of minimizing the value, the features whose regression coefficients are not 0 are important features.
[0028] In step S4, the generalized additive model is used To analyze and predict the relationship between variables, the model formula is usually expressed as:
[0029] in, represents the dependent variable, which is the target variable to be predicted; is the intercept term, which means that when all characteristic variables The expected target variable value when all values are 0; , , , is a nonlinear function used to represent the nonlinear relationship between the feature vector and the target vector; represents the error term, which is the random part that the model cannot explain; The training set data is used to fit the GAM model; the mean square error value MSE obtained from the test set is used to evaluate the model.
[0030] In step S5, the known environmental variables represent the collected Characteristic variables, the variables collected during the collection process do not include the target variable, that is, the number of species; after step S1 S4 achieves the purpose of predicting the target variable by using the characteristic variables.
[0031] In step S5, the prediction of the area to be measured includes the following preparations: S8-1. Collect historical environmental variables in the area; S8-2, predict the future environmental variables of the region; S8-3, select the environmental characteristic variables at the required time nodes to predict the suitability of the area; In step S8-1, by The climate database obtains the historical environmental variable data of the region, and the acquired features are the same as the features collected in step S1 Consistent.
[0032] In step S8-2, the autoregressive integrated moving average model is used to For fitting, use the trained The model predicts the future environmental variables in the region by: cleaning and standardizing time series data, drawing time series graphs, making non-stationary data stationary through differences, and using autocorrelation functions. and partial autocorrelation function Figure determination and , use the number of differences in data to determine , use the determined parameters to fit Model and use of trained The model makes predictions about future values; Among them, the autoregressive order Determined Autoregression Partially determines the current value and the previous The relationship between the hysteresis values; by the number of differences Determined points The moving average order determines how many times the data should be differentiated to make it stationary. Determined Moving Average Partially determines the current value and the previous The relationship between the error terms; autoregression ,integral and moving average The three parts clearly The structure of the model; After the prediction is completed, the environmental characteristic variables of the required time nodes are selected and the results are obtained through step S1. S4 achieves the purpose of predicting the target variable by using the characteristic variables.
[0033] In this embodiment: The characteristic variables are: temperature (℃), humidity (%), light intensity (lux), food quality (g), soil pH, soil moisture (%), organic matter content (%); the target variables are: the number of species per unit area; The existing data set is as follows. The data set has 10 samples in total. Each of the first 7 columns corresponds to a feature variable, and the last column corresponds to the target variable: Sample 1: ; Sample 2: ; Sample 3: ; Sample 4: ; Sample 5: ; Sample 6: ; Sample 7: ; Sample 8: ; Sample 9: ; Sample 10: ; The samples to be tested are: ; ; ; The sample to be tested only contains characteristic variables, and the model is used to predict the target variable, that is, the number of species, through the characteristic variables; the characteristics of the sample to be tested are consistent with the sample characteristics; Use cross-validation to divide the sample data set into a training set and a test set; divide the data set into 5 subsets, each containing two samples; each time select one of the subsets as the test set and the remaining 4 subsets as the training set; repeat five times, and each subset will be used as a test set in turn; use Regression algorithm to screen important features; regularization parameter Set to 0.1 and use coordinate descent method to adjust the regression coefficient Envoy The value is minimized, and the regression coefficients of the 7 characteristic variables are: ; The important characteristics are: temperature (℃), humidity (%), light intensity (lux), food quality (g), organic matter content (%); Building a generalized additive model To train the feature variables and output the predicted value of each cross-validation: Training set sample: ;Test set samples: ; True values of samples 1 and 2: ; True values of samples 1 and 2: ;
[0034] Training set sample: ;Test set samples: ; True values of samples 1 and 2: ; True values of samples 1 and 2: ;
[0035] Training set sample: ;Test set samples: ; True values of samples 1 and 2: ; True values of samples 1 and 2: ;
[0036] Training set sample: ;Test set samples: ; True values of samples 1 and 2: ; True values of samples 1 and 2: ;
[0037] Training set sample: ;Test set samples: ; True values of samples 1 and 2: ; True values of samples 1 and 2: ;
[0038] The mean square error of the model is: ; Use the above model to predict the three samples to be tested and get the results: Characteristic variables of sample 1 to be tested: ; The model predicts the target variable: ; That is, under this environmental variable, the predicted number of fly maggots is ; Characteristic variables of sample 2 to be tested: ; The model predicts the target variable: ; That is, under this environmental variable, the predicted number of fly maggots is ; Characteristic variables of sample 3 to be tested: ; The model predicts the target variable: ; That is, under this environmental variable, the predicted number of fly maggots is .
[0039] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0040] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for predicting the environmental suitability of fly maggots, characterized in that: The method comprises the following steps: S1. Collect and organize environmental variable data and fly maggot quantity data under corresponding variables; S2, integrating the data collected in step S1 into a data set and dividing it into a training set and a test set; S3. Use The regression algorithm screens environmental variables; S4. Building a generalized additive model , train and evaluate the model; S5. Use the trained model to make predictions for known environmental variables or areas to be tested.
2. A method for predicting the environmental suitability of fly maggots according to claim 1, characterized in that: In step S1, the environmental variable data refers to the state of environmental factors related to the survival of fly maggots; the fly maggot quantity data refers to the number of fly maggots per unit area; wherein the environmental factor is a characteristic variable, which is expressed as , is the number of characteristic variables; the number of fly maggots is the target variable, expressed as ; The feature variables are used to predict the target variables.
3. A method for predicting the environmental suitability of fly maggots according to claim 1, characterized in that: In step S2, a cross-validation method is used to divide the training set and the test set; Divide the data set into equal parts , to carry out In the first experiment, the first set of data was selected as the test data, and the remaining The first set of data was used as training data; in the second experiment, the second set of data was selected as test data, and the remaining The data is used as training data... In this experiment, we selected The data is used as test data, and the rest The data is used as training data; the experiment obtains indivual Mean square error, average The mean square error value is the cross validation error.
4. A method for predicting the environmental suitability of fly maggots according to claim 3, characterized in that: The mean square error value formula is: in, represents the number of test set samples, represents the true value, Represents the predicted value; in regression analysis, the mean square error is used to measure the average square error between the predicted value and the actual value; from the formula, it can be seen that the value of the mean square error is a non-negative number; when the predicted value is consistent with the true value, the mean square error is 0, so when the mean square error is close to 0, the predicted value is close to the true value; similarly, the smaller the cross-validation error value, the better the model performance.
5. The method for predicting the environmental suitability of fly maggots according to claim 1, characterized in that: In step S3, use Regression algorithms identify important environmental variables; The regression algorithm adds restrictions on weights as a regularization term, placing limited weights on more important feature dimensions: in, is the number of samples, represents the true value, represents the predicted value, is the regularization parameter, for norm; Regularization parameter Controls the strength of regularization, The larger the value, the greater the penalty for model complexity, and more environmental variable coefficients are compressed to 0; Small value, The regression algorithm is close to the general linear regression algorithm; the environmental variables whose coefficients are not 0 are retained; as The value increases continuously, and the variables whose environmental variable coefficients become 0 later have a higher proportion of importance in the model; using the iterative fitting along the regularization path Linear Model , by specifying the length of the regularization path, that is, limiting The ratio of minimum to maximum values is automatically generated Or manually set the regularization parameter The value of controls the penalty on model complexity to change the number of important feature variables; The model regression coefficient vector can be expressed as ,in is the number of features; then The norm can be calculated as ; That is, sum up the absolute values of all regression coefficients; Adjusted regression coefficients make The value of is minimized to achieve the purpose of controlling the complexity of the model through regularization while considering the accuracy; the regression coefficient is adjusted by using the coordinate descent method make In the process of minimizing the value, the features whose regression coefficients are not 0 are important features.
6. A method for predicting the environmental suitability of fly maggots according to claim 4, characterized in that: In step S4, the generalized additive model is used The relationship between variables is analyzed and predicted, and the model formula is expressed as: in, represents the dependent variable, i.e. the target variable to be predicted; is the intercept term, which means that when all characteristic variables The expected target variable value when all values are 0; , , , It is a nonlinear function, which is used to express the nonlinear relationship between the feature variable and the target variable; represents the error term, which is the random part that the model cannot explain; Fitting using training set data The model was evaluated using the mean square error (MSE) obtained from the test set.
7. A method for predicting the environmental suitability of fly maggots according to claim 1, characterized in that: In step S5, the known environmental variables represent the collected Characteristic variables, the variables collected during the collection process do not include the target variable, that is, the number of species; after step S1 S4 achieves the purpose of predicting the target variable by using the characteristic variables.
8. A method for predicting the environmental suitability of fly maggots according to claim 1, characterized in that: In step S5, the prediction of the area to be measured includes the following preparations: S8-1. Collect historical environmental variables of the region; S8-2, predict the future environmental variables of the region; S8-3, select the environmental characteristic variables at the required time nodes to predict the suitability of the area; In step S8-1, by The climate database obtains the historical environmental variable data of the region, and the acquired features are the same as the features collected in step S1 Consistent.
9. A method for predicting the environmental suitability of fly maggots according to claim 8, characterized in that: In step S8-2, the autoregressive integrated moving average model is used to For fitting, use the trained The model predicts the future environmental variables in the region by: cleaning and standardizing time series data, drawing time series graphs, making non-stationary data stationary through differences, and using autocorrelation functions. and partial autocorrelation function Figure determination and , use the number of differences in data to determine , use the determined parameters to fit Model and use of trained The model makes predictions about future values; Among them, the autoregressive order Determined Autoregression Partially determines the current value and the previous The relationship between the hysteresis values; by the number of differences Determined points The moving average order determines how many times the data should be differentiated to make it stationary. Determined Moving Average Partially determines the current value and the previous The relationship between the error terms; autoregression ,integral and moving average The three parts clearly The structure of the model; After the prediction is completed, the environmental characteristic variables of the required time nodes are selected and the results are obtained through step S1. S4 achieves the purpose of predicting the target variable by using the characteristic variables.
Citation Information
Cited By
Method, device and system for monitoring suitable growth area of harmful insects in grassland, and storage medium
CN121959256A
