Tea garden drought prediction method and system fusing LightGBM and LSVM
By integrating the tea garden drought prediction method of LightGBM and LSVM, and utilizing sensor arrays and multidisciplinary analysis technology, a multimodal feature extraction and dual-branch prediction model was constructed, which solved the limitations of the single model and data fusion obstacles in tea garden drought prediction, and achieved efficient and accurate tea garden drought prediction and real-time visualization.
Patent Information
- Application Number
- CN202510782203.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-12
- Publication Date
- 2025-09-23
AI Technical Summary
Existing technologies for predicting tea garden droughts have limitations of a single model and data fusion barriers, resulting in insufficient timeliness, accuracy, and generalization capabilities of the predictions.
The fusion model of LightGBM and LSVM is adopted, combined with the data collected by the sensor array. The correlation of environmental factors is analyzed through the Limma algorithm, COX survival regression model, GLM generalized linear model and SHAP. The multimodal feature extraction layer, dual-branch prediction layer and fusion output layer are constructed to predict the drought in the tea garden.
It improves the timeliness, accuracy and generalization ability of tea garden drought forecasts, and provides high-precision and highly interpretable forecast results through dynamic weighting strategy fusion model output, and supports real-time visualization display.
Smart Images

Figure CN120688034A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of smart agriculture technology, and in particular to a tea garden drought prediction method and system integrating LightGBM and LSVM. Background Art
[0002] Drought has a significant impact on tea plant growth, physiological metabolism, tea quality, and yield. Drought stress can have at least the following effects on tea plants: 1. It causes tea leaves to become smaller and thinner, with shortened leaf length and width, fewer leaves, and dwarfed plants. 2. It increases the number of lateral roots, but restricts overall root development, causing root hairs to dry out and die, and reducing absorption capacity. 3. Long-term drought can lead to stagnant growth, slow new shoot growth, bud and leaf shrinkage, and even death. 4. It inhibits photosynthesis, resulting in a decrease in photosynthetic rate. 5. It can cause changes in the content of quality components in tea, such as caffeine, amino acids, and tea polyphenols. Therefore, it is necessary to predict drought conditions in tea plantations and issue early warnings based on these predictions to better manage tea plantations, reduce water consumption, optimize irrigation plans, and improve crop yields and soil fertility.
[0003] Although machine learning algorithms have made some progress in the field of agricultural early warning in recent years, they still have the following shortcomings in tea garden scenarios:
[0004] Limitations of single models: For example, support vector machines (SVMs) perform well in small sample scenarios, but are inefficient in processing high-dimensional features (such as input vectors containing 20+ meteorological and soil parameters), and improper kernel function selection can easily lead to overfitting. Although random forests (RFs) have the ability to assess feature importance, the model variance generated by the bagging strategy is large, and the prediction lags significantly at soil moisture mutation points.
[0005] Data fusion obstacles: Simple feature concatenation is often used, which fails to effectively coordinate the spatiotemporal heterogeneity of multi-source data, directly affecting the model's generalization ability.
[0006] Therefore, how to provide a tea garden drought prediction method and system that integrates LightGBM and LSVM to improve the timeliness, accuracy and generalization ability of tea garden drought prediction has become a technical problem that needs to be solved urgently. Summary of the Invention
[0007] The technical problem to be solved by the present invention is to provide a tea garden drought prediction method and system that integrates LightGBM and LSVM, so as to improve the timeliness, accuracy and generalization ability of tea garden drought prediction.
[0008] In a first aspect, the present invention provides a tea garden drought prediction method integrating LightGBM and LSVM, comprising the following steps:
[0009] Step S1, collecting a large amount of historical monitoring data from the tea garden through a sensor array, constructing a data set after preprocessing each of the historical monitoring data, and performing a sample expansion operation on the data set;
[0010] Step S2, analyzing the data set using the Limma algorithm, COX survival regression model, GLM generalized linear model, and SHAP to obtain an environmental factor correlation graph;
[0011] Step S3, creating a tea garden drought prediction model based on the sequentially connected multimodal feature extraction layer, the dual-branch prediction layer, and the fusion output layer, and setting a loss function of the tea garden drought prediction model;
[0012] The multimodal feature extraction layer is used to extract the core feature group from the input detection data through the environmental factor correlation graph; the dual-branch prediction layer is used to perform nonlinear modeling and high-dimensional linear classification on the core feature group through the LightGBM branch and the LSVM branch, and output the tree enhancement probability prediction value and the kernel support decision value; the fusion output layer is used to fuse the tree enhancement probability prediction value and the kernel support decision value, and output the tea garden drought prediction result;
[0013] Step S4, dividing the data set into a training set, a validation set, and a test set, training the tea garden drought prediction model using the training set and the loss function, validating the trained tea garden drought prediction model using the validation set, testing the verified tea garden drought prediction model using the test set, and deploying the tea garden drought prediction model that has passed the test;
[0014] Step S5: Real-time monitoring data is collected from the tea garden through a sensor array. After preprocessing the real-time monitoring data, the data is input into the deployed tea garden drought prediction model to obtain a tea garden drought prediction result, and the tea garden drought prediction result is displayed on a visualization interface built based on the Shiny and DynNom framework.
[0015] Furthermore, the step S1 is specifically as follows:
[0016] A large amount of historical monitoring data is collected from the tea garden through a sensor array including humidity sensors, light intensity sensors, temperature sensors, carbon dioxide sensors, air pressure sensors, particulate matter sensors, global solar radiation sensors, pH sensors, salinity sensors, conductivity sensors, rainfall sensors, wind speed and direction sensors, and positioners;
[0017] The historical monitoring data at least include maximum light intensity, average light intensity, sunshine duration, maximum temperature, minimum temperature, average temperature, average daytime temperature, average nighttime temperature, daily temperature difference, maximum air humidity, minimum air humidity, average relative humidity, average daytime air humidity, average nighttime air humidity, maximum carbon dioxide concentration, minimum carbon dioxide concentration, average carbon dioxide concentration, average daytime carbon dioxide concentration, average nighttime carbon dioxide concentration, maximum air pressure, minimum air pressure, average air pressure, average daytime air pressure, average nighttime air pressure, air pressure difference, daily average PM2.5, daily average PM10, maximum TBQ global radiation, minimum TBQ global radiation, daily average TBQ global radiation, maximum soil temperature, minimum soil temperature, daily average soil temperature, soil temperature difference, maximum soil pH, minimum soil pH, average soil pH, maximum soil salinity, minimum soil salinity, average soil salinity, maximum soil electrical conductivity, minimum soil electrical conductivity, average soil electrical conductivity, rainfall, wind direction, wind speed, sampling location and sampling time;
[0018] After filling missing values and repairing outliers for each of the historical monitoring data, each of the historical monitoring data is labeled with soil entropy, drought level, and time segment labels to complete the preprocessing of each of the historical monitoring data. A data set is constructed based on the preprocessed historical monitoring data, and a sample expansion operation is performed on the data set through an autoregressive model; the time segment label is the month of the sampling time;
[0019] The missing value filling is based on the K-nearest neighbor filling method; the outlier repair is based on the Z-score algorithm and the K-means clustering algorithm, specifically:
[0020] Perform Z-score standardization on each of the historical monitoring data, calculate the Z-score value of each of the historical monitoring data after Z-score standardization, and compare the Z-score value with a preset abnormality threshold to screen out abnormal values; cluster the historical monitoring data after Z-score standardization using a K-means clustering algorithm to obtain several cluster centers, and repair each abnormal value in turn based on the nearest cluster center.
[0021] Furthermore, the step S2 is specifically as follows:
[0022] Segmenting the data set based on time segmentation labels to obtain a number of data subsets, and screening significant factors between the data subsets whose change rates exceed a preset change threshold using the Limma algorithm;
[0023] Calculating the first drought risk level of each of the significant factors using a COX survival regression model, calculating the second drought risk level of each of the significant factors using a GLM generalized linear model, and cross-validating the first drought risk level and the second drought risk level to screen N environmental factors with the greatest impact on drought conditions under different time segment labels from each of the significant factors, where N is a positive integer;
[0024] The contribution of each of the screened environmental factors is calculated by SHAP, and an environmental factor correlation graph is constructed based on the screened environmental factors, contribution and time segment labels.
[0025] Furthermore, in step S3, the multimodal feature extraction layer is composed of a spatiotemporal encoding module and a feature screening module; the spatiotemporal encoding module is used to perform spatiotemporal encoding on the input monitoring data after standardization processing to obtain spatiotemporal cross features; the feature screening module is used to screen a core feature group from each spatiotemporal cross feature through an environmental factor correlation graph;
[0026] The dual-branch prediction layer is composed of a LightGBM branch and an LSVM branch; the LightGBM branch is used to mine feature interactions in the core feature group through a leaf-wise growth strategy to perform nonlinear modeling, and output a tree-enhanced probability prediction value representing the probability of drought occurrence; the LSVM branch is used to process the nonlinear boundary of the core feature group through the RBF kernel to perform high-dimensional linear classification, and output a kernel support decision value representing the classification confidence;
[0027] The fusion output layer is used to fuse the tree enhanced probability prediction value and the kernel support decision value through a dynamic weighting strategy, and output the tea garden drought prediction result containing the drought period, drought level, drought location and confidence.
[0028] Furthermore, the step S4 is specifically as follows:
[0029] Using a stratified sampling technique, the dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1. The tea garden drought prediction model is trained using the training set. During the training process, the hyperparameters of the tea garden drought prediction model, including at least a learning rate, a batch size, and a number of training rounds, are continuously optimized until the loss function converges or a preset early stopping condition is met;
[0030] The MAE, MSE and RMSE are calculated using the validation set to validate the trained tea garden drought prediction model. If the validation fails, the training set is expanded to continue training. If the validation passes, then:
[0031] The ROC curve and the calibration curve are drawn by the test set to test the tea garden drought prediction model that has passed the verification. If the test fails, the training set is expanded to continue training; if the test passes, the training is terminated.
[0032] In the second aspect, the present invention provides a tea garden drought prediction system integrating LightGBM and LSVM, including the following modules:
[0033] A data set construction module is used to collect a large amount of historical monitoring data from the tea garden through a sensor array, construct a data set after pre-processing each of the historical monitoring data, and perform a sample expansion operation on the data set;
[0034] An environmental factor correlation graph generation module is used to analyze the data set using the Limma algorithm, the COX survival regression model, the GLM generalized linear model, and SHAP to obtain an environmental factor correlation graph;
[0035] A tea garden drought prediction model creation module is used to create a tea garden drought prediction model based on the sequentially connected multimodal feature extraction layer, the dual-branch prediction layer, and the fusion output layer, and set the loss function of the tea garden drought prediction model;
[0036] The multimodal feature extraction layer is used to extract the core feature group from the input detection data through the environmental factor correlation graph; the dual-branch prediction layer is used to perform nonlinear modeling and high-dimensional linear classification on the core feature group through the LightGBM branch and the LSVM branch, and output the tree enhancement probability prediction value and the kernel support decision value; the fusion output layer is used to fuse the tree enhancement probability prediction value and the kernel support decision value, and output the tea garden drought prediction result;
[0037] a tea garden drought prediction model training module, configured to divide the data set into a training set, a validation set, and a test set; train the tea garden drought prediction model using the training set and a loss function; verify the trained tea garden drought prediction model using the validation set; test the verified tea garden drought prediction model using the test set; and deploy the tested tea garden drought prediction model;
[0038] The tea garden drought prediction module is used to collect real-time monitoring data from the tea garden through a sensor array, pre-process the real-time monitoring data, input the deployed tea garden drought prediction model to obtain the tea garden drought prediction results, and display the tea garden drought prediction results on a visualization interface built based on the Shiny and DynNom frameworks.
[0039] Furthermore, the dataset construction module is specifically used to:
[0040] A large amount of historical monitoring data is collected from the tea garden through a sensor array including humidity sensors, light intensity sensors, temperature sensors, carbon dioxide sensors, air pressure sensors, particulate matter sensors, global solar radiation sensors, pH sensors, salinity sensors, conductivity sensors, rainfall sensors, wind speed and direction sensors, and positioners;
[0041] The historical monitoring data at least include maximum light intensity, average light intensity, sunshine duration, maximum temperature, minimum temperature, average temperature, average daytime temperature, average nighttime temperature, daily temperature difference, maximum air humidity, minimum air humidity, average relative humidity, average daytime air humidity, average nighttime air humidity, maximum carbon dioxide concentration, minimum carbon dioxide concentration, average carbon dioxide concentration, average daytime carbon dioxide concentration, average nighttime carbon dioxide concentration, maximum air pressure, minimum air pressure, average air pressure, average daytime air pressure, average nighttime air pressure, air pressure difference, daily average PM2.5, daily average PM10, maximum TBQ global radiation, minimum TBQ global radiation, daily average TBQ global radiation, maximum soil temperature, minimum soil temperature, daily average soil temperature, soil temperature difference, maximum soil pH, minimum soil pH, average soil pH, maximum soil salinity, minimum soil salinity, average soil salinity, maximum soil electrical conductivity, minimum soil electrical conductivity, average soil electrical conductivity, rainfall, wind direction, wind speed, sampling location and sampling time;
[0042] After filling missing values and repairing outliers for each of the historical monitoring data, each of the historical monitoring data is labeled with soil entropy, drought level, and time segment labels to complete the preprocessing of each of the historical monitoring data. A data set is constructed based on the preprocessed historical monitoring data, and a sample expansion operation is performed on the data set through an autoregressive model; the time segment label is the month of the sampling time;
[0043] The missing value filling is based on the K-nearest neighbor filling method; the outlier repair is based on the Z-score algorithm and the K-means clustering algorithm, specifically:
[0044] Perform Z-score standardization on each of the historical monitoring data, calculate the Z-score value of each of the historical monitoring data after Z-score standardization, and compare the Z-score value with a preset abnormality threshold to screen out abnormal values; cluster the historical monitoring data after Z-score standardization using a K-means clustering algorithm to obtain several cluster centers, and repair each abnormal value in turn based on the nearest cluster center.
[0045] Furthermore, the environmental factor correlation graph generation module is specifically used to:
[0046] Segmenting the data set based on time segmentation labels to obtain a number of data subsets, and screening significant factors between the data subsets whose change rates exceed a preset change threshold using the Limma algorithm;
[0047] Calculating the first drought risk level of each of the significant factors using a COX survival regression model, calculating the second drought risk level of each of the significant factors using a GLM generalized linear model, and cross-validating the first drought risk level and the second drought risk level to screen N environmental factors with the greatest impact on drought conditions under different time segment labels from each of the significant factors, where N is a positive integer;
[0048] The contribution of each of the screened environmental factors is calculated by SHAP, and an environmental factor correlation graph is constructed based on the screened environmental factors, contribution and time segment labels.
[0049] Furthermore, in the tea garden drought prediction model creation module, the multimodal feature extraction layer is composed of a spatiotemporal encoding module and a feature screening module; the spatiotemporal encoding module is used to perform spatiotemporal encoding on the input monitoring data after standardization processing, and obtain spatiotemporal cross features; the feature screening module is used to screen the core feature group from each spatiotemporal cross feature through the environmental factor correlation diagram;
[0050] The dual-branch prediction layer is composed of a LightGBM branch and an LSVM branch; the LightGBM branch is used to mine feature interactions in the core feature group through a leaf-wise growth strategy to perform nonlinear modeling, and output a tree-enhanced probability prediction value representing the probability of drought occurrence; the LSVM branch is used to process the nonlinear boundary of the core feature group through the RBF kernel to perform high-dimensional linear classification, and output a kernel support decision value representing the classification confidence;
[0051] The fusion output layer is used to fuse the tree enhanced probability prediction value and the kernel support decision value through a dynamic weighting strategy, and output the tea garden drought prediction result containing the drought period, drought level, drought location and confidence.
[0052] Furthermore, the tea garden drought prediction model training module is specifically used to:
[0053] Using a stratified sampling technique, the dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1. The tea garden drought prediction model is trained using the training set. During the training process, the hyperparameters of the tea garden drought prediction model, including at least a learning rate, a batch size, and a number of training rounds, are continuously optimized until the loss function converges or a preset early stopping condition is met;
[0054] The MAE, MSE and RMSE are calculated using the validation set to validate the trained tea garden drought prediction model. If the validation fails, the training set is expanded to continue training. If the validation passes, then:
[0055] The ROC curve and the calibration curve are drawn by the test set to test the tea garden drought prediction model that has passed the verification. If the test fails, the training set is expanded to continue training; if the test passes, the training is terminated.
[0056] The advantages of the present invention are:
[0057] 1. A large amount of historical monitoring data is collected from the tea garden through the sensor array to construct a data set and perform sample expansion operations. Then, the data set is analyzed by the Limma algorithm, COX survival regression model, GLM generalized linear model and SHAP to obtain the environmental factor correlation graph; then, a tea garden drought prediction model is created based on the sequentially connected multimodal feature extraction layer, dual-branch prediction layer and fusion output layer, and the loss function of the tea garden drought prediction model is set; the multimodal feature extraction layer is used to extract the core feature group from the input detection data through the environmental factor correlation graph; the dual-branch prediction layer is used to perform nonlinear modeling and high-dimensional linear classification on the core feature group through the LightGBM branch and the LSVM branch, and output the tree enhanced probability prediction value and the kernel support decision value; the fusion output layer is used to fuse the tree enhanced probability prediction value and the kernel support decision value, and output the tea garden drought prediction result; then the data set is divided into training set, validation set and test set, the tea garden drought prediction model is trained by the training set and the loss function, the trained tea garden drought prediction model is verified by the validation set, and the validation set is verified by the test set. The tea garden drought prediction model that has passed the test is tested and deployed. Finally, real-time monitoring data is collected from the tea garden through a sensor array. After preprocessing, the real-time monitoring data is input into the tea garden drought prediction model to obtain the tea garden drought prediction results, which are then displayed on a visualization interface built based on the Shiny and DynNom frameworks. In other words, the tea garden drought prediction model is used to predict tea garden drought. The tea garden drought prediction model combines LightGBM and LSVM, effectively integrating the advantages of the two. Compared with SVM, LSVM has the advantages of short training time, low computational complexity, suitability for large-scale data sets, and strong generalization ability. The fusion output layer is used to fuse the tree enhancement probability prediction value and the kernel support decision value through a dynamic weighting strategy, replacing the traditional simple feature splicing. The environmental factor correlation map is combined to select and adjust the weight of each feature, that is, the weights of the features corresponding to the N environmental factors that have the greatest impact on drought prediction under the current time segment label are increased, ultimately greatly improving the timeliness, accuracy, and generalization ability of tea garden drought prediction.
[0058] 2. By adopting an array of 13 types of sensors (covering multiple dimensions such as meteorology, soil, and environment), more than 40 environmental parameters are collected to ensure that the data covers all elements of the tea garden ecosystem and avoid single-dimensional bias; by combining K-nearest neighbor filling (preserving local features) and Z-score+K-means outlier repair (dynamic clustering to correct global distribution), data quality is improved, which is better than traditional mean filling or simple filtering methods; by introducing time segment labels (months) and spatial location labels, and expanding the sample through the autoregressive model, the sample imbalance problem caused by seasonal fluctuations in agricultural data is effectively solved.
[0059] 3. The Limma algorithm (biostatistics) is used to screen significant factors, and the COX survival model (risk analysis) and GLM (generalized linear regression) are used to calculate the risk level. Combined with SHAP (explainable AI) to quantify the contribution, dual verification of statistical models and machine learning is achieved, so that the selected environmental factors have both statistical significance and practical explanatory power, avoiding overfitting or spurious correlation interference.
[0060] 4. Through the standardized processing of the spatiotemporal coding module and the generation of spatiotemporal cross-features (such as the combination of "daytime humidity-nighttime temperature difference"), the dynamic evolution of the tea garden microenvironment can be effectively captured; through the feature screening module, directional filtering is performed based on the environmental factor correlation map to reduce redundant feature interference.
[0061] 5. Through the LightGBM branch based on the leaf-wise growth strategy, feature interactions (such as the nonlinear relationship between "soil conductivity and rainfall") are efficiently mined and probability predictions are output; through the RBF kernel function of the LSVM branch, high-dimensional space is mapped to solve complex boundary classification problems (such as fuzzy scenarios of drought level thresholds), and decision confidence is output. The dynamic weighted fusion strategy is used to adaptively adjust the weights of the two branches (such as LSVM confidence is mainly used in the early stage of drought, and LightGBM probability is mainly used in the persistent period), effectively improving the robustness of the prediction.
[0062] 6. The sample distribution characteristics are retained by dividing the data set into layers at a ratio of 8:1:1; overfitting is prevented by setting dual conditions of loss function convergence and early stopping; MAE (mean absolute error), MSE (mean square error), and RMSE (root mean square error) are used to evaluate the accuracy of numerical predictions in the verification phase; ROC curve (discrimination) and calibration curve (probability calibration) are used for dual verification in the testing phase to ensure the reliability of the model in actual scenarios; the training set is dynamically expanded when verification / testing fails, forming a "training-feedback" closed loop to continuously optimize the model's generalization ability.
[0063] 7. The dynamic visualization interface built through the Shiny+DynNom framework supports multi-dimensional display of drought level, spatial location, and confidence level, making it easier for agricultural managers to quickly locate high-risk areas and formulate irrigation strategies.
[0064] 8. By innovatively applying the Limma algorithm used for differential gene expression in biostatistics to environmental factor screening, we can identify environmental factors that change significantly under time segment labels through linear model fitting and Bayesian correction, thereby improving the statistical rigor of feature screening and avoiding the misjudgment of time series non-independent data by traditional analysis of variance (ANOVA).
[0065] 9. To address seasonal fluctuations in agricultural data (such as the imbalance in data volume between the rainy season and the dry season), an autoregressive model (AR) is used to generate synthetic data that conforms to the laws of time series, rather than simple oversampling, to avoid introducing noise and enhance the model's ability to predict rare climate events (such as extreme droughts).
[0066] 10. By combining month (time) and sampling location (space) labels, we can expand the data in a targeted manner, for example, simulating the changes in soil parameters at the same location in different months, and enhancing the model's adaptability to spatiotemporal heterogeneity.
[0067] 11. By integrating multi-dimensional sensor data with interdisciplinary analysis methods (Limma algorithm, COX model, GLM and SHAP), and combining the advantages of LightGBM and LSVM dual-branch models, a tea garden drought prediction framework with high precision and strong interpretability was constructed; by adopting spatiotemporal coding and dynamic feature screening mechanisms to accurately extract core environmental factors, and optimizing data quality through autoregressive data enhancement and Z-score+K-means anomaly repair, the dynamic weighted fusion of dual models was used to improve the robustness of prediction, and the spatiotemporal visualization of drought was achieved based on Shiny+DynNom, ultimately forming a closed-loop system from data collection, model training to real-time early warning, which significantly solved the problems of data sparsity, model generalization and decision credibility in agricultural scenarios, and provided efficient and scalable technical support for intelligent drought management in tea gardens. BRIEF DESCRIPTION OF THE DRAWINGS
[0068] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0069] Figure 1 This is a flowchart of a tea garden drought prediction method that integrates LightGBM and LSVM in the present invention.
[0070] Figure 2 This is a structural diagram of a tea garden drought prediction system that integrates LightGBM and LSVM in the present invention. DETAILED DESCRIPTION
[0071] The technical solution in the embodiment of the present application has the following overall idea: tea garden drought is predicted through a pre-trained tea garden drought prediction model. The tea garden drought prediction model combines LightGBM and LSVM, effectively integrating the advantages of the two. Compared with SVM, LSVM has the advantages of short training time, small computational complexity, suitability for large-scale data sets and strong generalization ability. The fusion output layer is used to fuse the tree enhanced probability prediction value and the kernel support decision value through a dynamic weighting strategy, instead of the traditional simple feature splicing, and select and adjust the weight of each feature in combination with the environmental factor correlation graph, that is, increase the weight of the features corresponding to the N environmental factors that have the greatest impact on drought prediction under the current time segment label, thereby improving the timeliness, accuracy and generalization ability of tea garden drought prediction.
[0072] Please refer to Figures 1 to 2 As shown, a preferred embodiment of the tea garden drought prediction method integrating LightGBM and LSVM of the present invention includes the following steps:
[0073] Step S1, collecting a large amount of historical monitoring data from the tea garden through a sensor array, constructing a data set after preprocessing each of the historical monitoring data, and performing a sample expansion operation on the data set;
[0074] Step S2, analyzing the data set using the Limma algorithm, COX survival regression model, GLM generalized linear model, and SHAP to obtain an environmental factor correlation graph;
[0075] Step S3, creating a tea garden drought prediction model based on the sequentially connected multimodal feature extraction layer, the dual-branch prediction layer, and the fusion output layer, and setting a loss function of the tea garden drought prediction model;
[0076] The multimodal feature extraction layer is used to extract the core feature group from the input detection data through the environmental factor correlation graph; the dual-branch prediction layer is used to perform nonlinear modeling and high-dimensional linear classification on the core feature group through the LightGBM branch and the LSVM branch, and output the tree enhancement probability prediction value and the kernel support decision value; the fusion output layer is used to fuse the tree enhancement probability prediction value and the kernel support decision value, and output the tea garden drought prediction result;
[0077] Step S4, dividing the data set into a training set, a validation set, and a test set, training the tea garden drought prediction model using the training set and the loss function, validating the trained tea garden drought prediction model using the validation set, testing the verified tea garden drought prediction model using the test set, and deploying the tea garden drought prediction model that has passed the test;
[0078] Step S5: Real-time monitoring data is collected from the tea garden via a sensor array. After preprocessing, the data is input into the deployed tea garden drought prediction model to obtain a tea garden drought prediction result. The tea garden drought prediction result is then displayed on a visualization interface built using the Shiny and DynNom frameworks. This dynamic visualization interface, built using the Shiny + DynNom frameworks, supports multi-dimensional display of drought severity, spatial location, and confidence level, enabling agricultural managers to quickly locate high-risk areas and formulate irrigation strategies.
[0079] By integrating multi-dimensional sensor data with interdisciplinary analysis methods (Limma algorithm, COX model, GLM and SHAP), and combining the advantages of LightGBM and LSVM dual-branch models, a tea garden drought prediction framework with both high precision and strong interpretability was constructed; by adopting spatiotemporal coding and dynamic feature screening mechanisms to accurately extract core environmental factors, and optimizing data quality through autoregressive data enhancement and Z-score+K-means anomaly repair, the dynamic weighted fusion of dual models was used to improve prediction robustness, and drought spatiotemporal visualization was achieved based on Shiny+DynNom, ultimately forming a closed-loop system from data collection, model training to real-time early warning, which significantly solved the problems of data sparsity, model generalization and decision credibility in agricultural scenarios, and provided efficient and scalable technical support for intelligent drought management in tea gardens.
[0080] The step S1 is specifically as follows:
[0081] A large amount of historical monitoring data is collected from the tea garden through a sensor array including humidity sensors, light intensity sensors, temperature sensors, carbon dioxide sensors, air pressure sensors, particulate matter sensors, global solar radiation sensors, pH sensors, salinity sensors, conductivity sensors, rainfall sensors, wind speed and direction sensors, and positioners;
[0082] The historical monitoring data at least include maximum light intensity, average light intensity, sunshine duration, maximum temperature, minimum temperature, average temperature, average daytime temperature, average nighttime temperature, daily temperature difference, maximum air humidity, minimum air humidity, average relative humidity, average daytime air humidity, average nighttime air humidity, maximum carbon dioxide concentration, minimum carbon dioxide concentration, average carbon dioxide concentration, average daytime carbon dioxide concentration, average nighttime carbon dioxide concentration, maximum air pressure, minimum air pressure, average air pressure, average daytime air pressure, average nighttime air pressure, air pressure difference, daily average PM2.5, daily average PM10, maximum TBQ global radiation, minimum TBQ global radiation, daily average TBQ global radiation, maximum soil temperature, minimum soil temperature, daily average soil temperature, soil temperature difference, maximum soil pH, minimum soil pH, average soil pH, maximum soil salinity, minimum soil salinity, average soil salinity, maximum soil electrical conductivity, minimum soil electrical conductivity, average soil electrical conductivity, rainfall, wind direction, wind speed, sampling location and sampling time;
[0083] After filling missing values and repairing outliers for each of the historical monitoring data, each of the historical monitoring data is labeled with soil entropy, drought level, and time segment labels to complete the preprocessing of each of the historical monitoring data. A data set is constructed based on the preprocessed historical monitoring data, and a sample expansion operation is performed on the data set through an autoregressive model; the time segment label is the month of the sampling time;
[0084] The missing value filling is based on the K-nearest neighbor filling method; the outlier repair is based on the Z-score algorithm and the K-means clustering algorithm, specifically:
[0085] Perform Z-score standardization on each of the historical monitoring data, calculate the Z-score value of each of the historical monitoring data after Z-score standardization, and compare the Z-score value with a preset abnormality threshold to screen out abnormal values; cluster the historical monitoring data after Z-score standardization using a K-means clustering algorithm to obtain several cluster centers, and repair each abnormal value in turn based on the nearest cluster center.
[0086] By adopting an array of 13 types of sensors (covering multiple dimensions such as meteorology, soil, and environment), more than 40 environmental parameters are collected to ensure that the data covers all elements of the tea garden ecosystem and avoid single-dimensional bias; by combining K-nearest neighbor filling (retaining local features) and Z-score+K-means outlier repair (dynamic clustering to correct global distribution), data quality is improved, which is better than traditional mean filling or simple filtering methods; by introducing time segment labels (months) and spatial location labels, and expanding samples through autoregressive models, the sample imbalance problem caused by seasonal fluctuations in agricultural data is effectively solved.
[0087] In response to seasonal fluctuations in agricultural data (such as the imbalance in data volume between the rainy season and the dry season), an autoregressive model (AR) is used to generate synthetic data that conforms to the laws of time series, rather than simple oversampling, to avoid introducing noise and enhance the model's ability to predict rare climate events (such as extreme droughts).
[0088] By combining month (time) and sampling location (space) labels to expand data in a targeted manner, for example, simulating the changes in soil parameters at the same location in different months, the model's adaptability to spatiotemporal heterogeneity is enhanced.
[0089] The step S2 is specifically as follows:
[0090] Segmenting the data set based on time segmentation labels to obtain a number of data subsets, and screening significant factors between the data subsets whose change rates exceed a preset change threshold using the Limma algorithm;
[0091] Calculating the first drought risk level of each of the significant factors using a COX survival regression model, calculating the second drought risk level of each of the significant factors using a GLM generalized linear model, and cross-validating the first drought risk level and the second drought risk level to screen N environmental factors with the greatest impact on drought conditions under different time segment labels from each of the significant factors, where N is a positive integer;
[0092] The contribution of each selected environmental factor was calculated using SHAP, and a correlation graph of these factors was constructed based on the selected environmental factors, their contribution, and the time segment labels. The introduction of the Feature Contribution Explanation (SHAP) algorithm breaks through the "black box" limitations of traditional machine learning models. By combining SHAP values with the factor contribution analysis mechanism, the specific contribution of each factor in the drought prediction model is accurately revealed.
[0093] Significant factors are screened through the Limma algorithm (biostatistics), and the risk level is calculated using the COX survival model (risk analysis) and GLM (generalized linear regression) dual paths. Combined with SHAP (explainable AI) to quantify the contribution, dual verification of statistical models and machine learning is achieved, so that the screened environmental factors have both statistical significance and practical explanatory power, avoiding overfitting or pseudo-correlation interference.
[0094] By innovatively applying the Limma algorithm used for differential gene expression in biostatistics to environmental factor screening, and through linear model fitting and Bayesian correction, we can identify environmental factors that change significantly under time segment labels, improve the statistical rigor of feature screening, and avoid the misjudgment of time series non-independent data by traditional analysis of variance (ANOVA).
[0095] In step S3, the multimodal feature extraction layer is composed of a spatiotemporal encoding module and a feature screening module; the spatiotemporal encoding module is used to perform spatiotemporal encoding on the input monitoring data after standardization processing to obtain spatiotemporal cross features; the feature screening module is used to screen the core feature group from each spatiotemporal cross feature through the environmental factor correlation diagram;
[0096] Through the standardized processing of the spatiotemporal coding module and the generation of spatiotemporal cross-features (such as the combination of "daytime humidity-nighttime temperature difference"), the dynamic evolution of the tea garden microenvironment can be effectively captured; through the feature screening module, directional filtering is performed based on the environmental factor correlation map to reduce redundant feature interference.
[0097] The dual-branch prediction layer is composed of a LightGBM branch and an LSVM branch; the LightGBM branch is used to mine feature interactions in the core feature group through a leaf-wise growth strategy to perform nonlinear modeling, and output a tree-enhanced probability prediction value representing the probability of drought occurrence; the LSVM branch is used to process the nonlinear boundary of the core feature group through the RBF kernel to perform high-dimensional linear classification, and output a kernel support decision value representing the classification confidence;
[0098] Through the LightGBM branch based on the leaf-wise growth strategy, feature interactions (such as the nonlinear relationship between "soil conductivity and rainfall") are efficiently mined to output probability predictions; through the RBF kernel function of the LSVM branch, high-dimensional space is mapped to solve complex boundary classification problems (such as fuzzy scenarios of drought level thresholds), and decision confidence is output. The dynamic weighted fusion strategy is used to adaptively adjust the weights of the two branches (such as LSVM confidence is mainly used in the early stage of drought, and LightGBM probability is mainly used in the persistent period), effectively improving the robustness of the prediction.
[0099] The fusion output layer is used to fuse the tree-enhanced probability prediction value and the kernel support decision value through a dynamic weighting strategy, outputting a tea garden drought prediction result that includes the drought period, drought level, drought location, and confidence level. Specifically, a fully connected layer + Softmax is used to output the probability of drought level (level 1, level 2, level 3, and level 4), the probability of drought period, the probability of drought location, and the confidence level. Temperature scaling technology is also introduced to calibrate the prediction confidence level.
[0100] The formula of the loss function is:
[0101] L_total=α·L_lgb+β·L_lsvm+γ·L_fuse;
[0102] Among them, L_total represents the loss value of the loss function; α, β and γ all represent weight coefficients, α+β+γ=1, γ>α,β; L_lgb represents the binary cross entropy loss of the LightGBM branch, which is used to measure the difference between the tree enhanced probability prediction value p and the true label y; L_lsvm represents the Hinge Loss of the LSVM branch, which is used to constrain the sign consistency of the kernel support decision value d and the true label y; L_fuse represents the cross entropy loss of the fusion output, which is used to optimize the complementary fusion of the dual-branch prediction results.
[0103] The triple constraints are implemented through the loss function to effectively prevent single-branch overfitting and improve the generalization of drought prediction: ① retaining LightGBM's modeling ability for complex feature interactions; ② strengthening LSVM's robust classification performance in high-dimensional space; ③ through the weight distribution of γ>α,β, forcing the model to prioritize the optimization of the fusion result.
[0104] The step S4 is specifically as follows:
[0105] The data set is divided into a training set, a validation set, and a test set in a ratio of 8:1:1 using a stratified sampling technique. A tea garden drought prediction model is trained using the training set. During the training process, the hyperparameters of the tea garden drought prediction model, including at least the learning rate, batch size, and number of training rounds, are continuously optimized until the loss function converges or a preset early stopping condition is met. The AdamW optimizer is used during the training process, combined with cosine annealing learning rate scheduling;
[0106] The MAE, MSE and RMSE are calculated using the validation set to validate the trained tea garden drought prediction model. If the validation fails, the training set is expanded to continue training. If the validation passes, then:
[0107] The ROC curve and the calibration curve are drawn by the test set to test the tea garden drought prediction model that has passed the verification. If the test fails, the training set is expanded to continue training; if the test passes, the training is terminated.
[0108] The data set is divided into layers at a ratio of 8:1:1 to preserve the sample distribution characteristics; overfitting is prevented by setting dual conditions of loss function convergence and early stopping; MAE (mean absolute error), MSE (mean square error), and RMSE (root mean square error) are used to evaluate the accuracy of numerical predictions in the verification phase; ROC curve (discrimination) and calibration curve (probability calibration) are used for dual verification in the testing phase to ensure the reliability of the model in actual scenarios; and the training set is dynamically expanded when verification / testing fails to form a "training-feedback" closed loop to continuously optimize the model's generalization ability.
[0109] The ROC curve is used to evaluate the discriminative ability of the tea plantation drought prediction model. The area under the ROC curve (AUC) measures the model's efficiency. An ideal model's ROC curve is as close to the upper left corner as possible, indicating a high TPR and a low FPR, demonstrating excellent discriminative ability. The AUC quantifies the model's overall discriminative ability. In statistical analysis, an AUC value greater than 0.7 is considered to indicate excellent discriminative performance. Higher AUC values indicate a stronger model's ability to distinguish between positive and negative classes.
[0110] A calibration curve is a method used to assess the accuracy of a disease risk model in predicting the probability of future occurrence. In the present invention, a calibration curve is used to assess the degree of fit between the drought level changes in a tea garden under different key factor parameters and the actual values.
[0111] A preferred embodiment of the tea garden drought prediction system integrating LightGBM and LSVM in the present invention includes the following modules:
[0112] A data set construction module is used to collect a large amount of historical monitoring data from the tea garden through a sensor array, construct a data set after pre-processing each of the historical monitoring data, and perform a sample expansion operation on the data set;
[0113] An environmental factor correlation graph generation module is used to analyze the data set using the Limma algorithm, the COX survival regression model, the GLM generalized linear model, and SHAP to obtain an environmental factor correlation graph;
[0114] A tea garden drought prediction model creation module is used to create a tea garden drought prediction model based on the sequentially connected multimodal feature extraction layer, the dual-branch prediction layer, and the fusion output layer, and set the loss function of the tea garden drought prediction model;
[0115] The multimodal feature extraction layer is used to extract the core feature group from the input detection data through the environmental factor correlation graph; the dual-branch prediction layer is used to perform nonlinear modeling and high-dimensional linear classification on the core feature group through the LightGBM branch and the LSVM branch, and output the tree enhancement probability prediction value and the kernel support decision value; the fusion output layer is used to fuse the tree enhancement probability prediction value and the kernel support decision value, and output the tea garden drought prediction result;
[0116] a tea garden drought prediction model training module, configured to divide the data set into a training set, a validation set, and a test set; train the tea garden drought prediction model using the training set and a loss function; verify the trained tea garden drought prediction model using the validation set; test the verified tea garden drought prediction model using the test set; and deploy the tested tea garden drought prediction model;
[0117] The tea garden drought prediction module collects real-time monitoring data from tea gardens using a sensor array. After preprocessing this data, it is input into the deployed tea garden drought prediction model to generate drought prediction results. These results are then displayed on a visualization interface built using the Shiny and DynNom frameworks. This dynamic visualization interface, built using the Shiny and DynNom frameworks, supports multi-dimensional display of drought severity, spatial location, and confidence levels, enabling agricultural managers to quickly locate high-risk areas and formulate irrigation strategies.
[0118] By integrating multi-dimensional sensor data with interdisciplinary analysis methods (Limma algorithm, COX model, GLM and SHAP), and combining the advantages of LightGBM and LSVM dual-branch models, a tea garden drought prediction framework with both high precision and strong interpretability was constructed; by adopting spatiotemporal coding and dynamic feature screening mechanisms to accurately extract core environmental factors, and optimizing data quality through autoregressive data enhancement and Z-score+K-means anomaly repair, the dynamic weighted fusion of dual models was used to improve prediction robustness, and drought spatiotemporal visualization was achieved based on Shiny+DynNom, ultimately forming a closed-loop system from data collection, model training to real-time early warning, which significantly solved the problems of data sparsity, model generalization and decision credibility in agricultural scenarios, and provided efficient and scalable technical support for intelligent drought management in tea gardens.
[0119] The dataset construction module is specifically used for:
[0120] A large amount of historical monitoring data is collected from the tea garden through a sensor array including humidity sensors, light intensity sensors, temperature sensors, carbon dioxide sensors, air pressure sensors, particulate matter sensors, global solar radiation sensors, pH sensors, salinity sensors, conductivity sensors, rainfall sensors, wind speed and direction sensors, and positioners;
[0121] The historical monitoring data at least include maximum light intensity, average light intensity, sunshine duration, maximum temperature, minimum temperature, average temperature, average daytime temperature, average nighttime temperature, daily temperature difference, maximum air humidity, minimum air humidity, average relative humidity, average daytime air humidity, average nighttime air humidity, maximum carbon dioxide concentration, minimum carbon dioxide concentration, average carbon dioxide concentration, average daytime carbon dioxide concentration, average nighttime carbon dioxide concentration, maximum air pressure, minimum air pressure, average air pressure, average daytime air pressure, average nighttime air pressure, air pressure difference, daily average PM2.5, daily average PM10, maximum TBQ global radiation, minimum TBQ global radiation, daily average TBQ global radiation, maximum soil temperature, minimum soil temperature, daily average soil temperature, soil temperature difference, maximum soil pH, minimum soil pH, average soil pH, maximum soil salinity, minimum soil salinity, average soil salinity, maximum soil electrical conductivity, minimum soil electrical conductivity, average soil electrical conductivity, rainfall, wind direction, wind speed, sampling location and sampling time;
[0122] After filling missing values and repairing outliers for each of the historical monitoring data, each of the historical monitoring data is labeled with soil entropy, drought level, and time segment labels to complete the preprocessing of each of the historical monitoring data. A data set is constructed based on the preprocessed historical monitoring data, and a sample expansion operation is performed on the data set through an autoregressive model; the time segment label is the month of the sampling time;
[0123] The missing value filling is based on the K-nearest neighbor filling method; the outlier repair is based on the Z-score algorithm and the K-means clustering algorithm, specifically:
[0124] Perform Z-score standardization on each of the historical monitoring data, calculate the Z-score value of each of the historical monitoring data after Z-score standardization, and compare the Z-score value with a preset abnormality threshold to screen out abnormal values; cluster the historical monitoring data after Z-score standardization using a K-means clustering algorithm to obtain several cluster centers, and repair each abnormal value in turn based on the nearest cluster center.
[0125] By adopting an array of 13 types of sensors (covering multiple dimensions such as meteorology, soil, and environment), more than 40 environmental parameters are collected to ensure that the data covers all elements of the tea garden ecosystem and avoid single-dimensional bias; by combining K-nearest neighbor filling (retaining local features) and Z-score+K-means outlier repair (dynamic clustering to correct global distribution), data quality is improved, which is better than traditional mean filling or simple filtering methods; by introducing time segment labels (months) and spatial location labels, and expanding samples through autoregressive models, the sample imbalance problem caused by seasonal fluctuations in agricultural data is effectively solved.
[0126] In response to seasonal fluctuations in agricultural data (such as the imbalance in data volume between the rainy season and the dry season), an autoregressive model (AR) is used to generate synthetic data that conforms to the laws of time series, rather than simple oversampling, to avoid introducing noise and enhance the model's ability to predict rare climate events (such as extreme droughts).
[0127] By combining month (time) and sampling location (space) labels to expand data in a targeted manner, for example, simulating the changes in soil parameters at the same location in different months, the model's adaptability to spatiotemporal heterogeneity is enhanced.
[0128] The environmental factor correlation graph generation module is specifically used for:
[0129] Segmenting the data set based on time segmentation labels to obtain a number of data subsets, and screening significant factors between the data subsets whose change rates exceed a preset change threshold using the Limma algorithm;
[0130] Calculating the first drought risk level of each of the significant factors using a COX survival regression model, calculating the second drought risk level of each of the significant factors using a GLM generalized linear model, and cross-validating the first drought risk level and the second drought risk level to screen N environmental factors with the greatest impact on drought conditions under different time segment labels from each of the significant factors, where N is a positive integer;
[0131] The contribution of each selected environmental factor was calculated using SHAP, and a correlation graph of these factors was constructed based on the selected environmental factors, their contribution, and the time segment labels. The introduction of the Feature Contribution Explanation (SHAP) algorithm breaks through the "black box" limitations of traditional machine learning models. By combining SHAP values with the factor contribution analysis mechanism, the specific contribution of each factor in the drought prediction model is accurately revealed.
[0132] Significant factors are screened through the Limma algorithm (biostatistics), and the risk level is calculated using the COX survival model (risk analysis) and GLM (generalized linear regression) dual paths. Combined with SHAP (explainable AI) to quantify the contribution, dual verification of statistical models and machine learning is achieved, so that the screened environmental factors have both statistical significance and practical explanatory power, avoiding overfitting or pseudo-correlation interference.
[0133] By innovatively applying the Limma algorithm used for differential gene expression in biostatistics to environmental factor screening, and through linear model fitting and Bayesian correction, we can identify environmental factors that change significantly under time segment labels, improve the statistical rigor of feature screening, and avoid the misjudgment of time series non-independent data by traditional analysis of variance (ANOVA).
[0134] In the tea garden drought prediction model creation module, the multimodal feature extraction layer is composed of a spatiotemporal encoding module and a feature screening module; the spatiotemporal encoding module is used to perform spatiotemporal encoding on the input monitoring data after standardization processing to obtain spatiotemporal cross features; the feature screening module is used to screen the core feature group from each spatiotemporal cross feature through the environmental factor correlation diagram;
[0135] Through the standardized processing of the spatiotemporal coding module and the generation of spatiotemporal cross-features (such as the combination of "daytime humidity-nighttime temperature difference"), the dynamic evolution of the tea garden microenvironment can be effectively captured; through the feature screening module, directional filtering is performed based on the environmental factor correlation map to reduce redundant feature interference.
[0136] The dual-branch prediction layer is composed of a LightGBM branch and an LSVM branch; the LightGBM branch is used to mine feature interactions in the core feature group through a leaf-wise growth strategy to perform nonlinear modeling, and output a tree-enhanced probability prediction value representing the probability of drought occurrence; the LSVM branch is used to process the nonlinear boundary of the core feature group through the RBF kernel to perform high-dimensional linear classification, and output a kernel support decision value representing the classification confidence;
[0137] Through the LightGBM branch based on the leaf-wise growth strategy, feature interactions (such as the nonlinear relationship between "soil conductivity and rainfall") are efficiently mined to output probability predictions; through the RBF kernel function of the LSVM branch, high-dimensional space is mapped to solve complex boundary classification problems (such as fuzzy scenarios of drought level thresholds), and decision confidence is output. The dynamic weighted fusion strategy is used to adaptively adjust the weights of the two branches (such as LSVM confidence is mainly used in the early stage of drought, and LightGBM probability is mainly used in the persistent period), effectively improving the robustness of the prediction.
[0138] The fusion output layer is used to fuse the tree-enhanced probability prediction value and the kernel support decision value through a dynamic weighting strategy, outputting a tea garden drought prediction result that includes the drought period, drought level, drought location, and confidence level. Specifically, a fully connected layer + Softmax is used to output the probability of drought level (level 1, level 2, level 3, and level 4), the probability of drought period, the probability of drought location, and the confidence level. Temperature scaling technology is also introduced to calibrate the prediction confidence level.
[0139] The formula of the loss function is:
[0140] L_total=α·L_lgb+β·L_lsvm+γ·L_fuse;
[0141] Among them, L_total represents the loss value of the loss function; α, β and γ all represent weight coefficients, α+β+γ=1, γ>α,β; L_lgb represents the binary cross entropy loss of the LightGBM branch, which is used to measure the difference between the tree enhanced probability prediction value p and the true label y; L_lsvm represents the Hinge Loss of the LSVM branch, which is used to constrain the sign consistency of the kernel support decision value d and the true label y; L_fuse represents the cross entropy loss of the fusion output, which is used to optimize the complementary fusion of the dual-branch prediction results.
[0142] The triple constraints are implemented through the loss function to effectively prevent single-branch overfitting and improve the generalization of drought prediction: ① retaining LightGBM's modeling ability for complex feature interactions; ② strengthening LSVM's robust classification performance in high-dimensional space; ③ through the weight distribution of γ>α,β, forcing the model to prioritize the optimization of the fusion result.
[0143] The tea garden drought prediction model training module is specifically used for:
[0144] The data set is divided into a training set, a validation set, and a test set in a ratio of 8:1:1 using a stratified sampling technique. A tea garden drought prediction model is trained using the training set. During the training process, the hyperparameters of the tea garden drought prediction model, including at least the learning rate, batch size, and number of training rounds, are continuously optimized until the loss function converges or a preset early stopping condition is met. The AdamW optimizer is used during the training process, combined with cosine annealing learning rate scheduling;
[0145] The MAE, MSE and RMSE are calculated using the validation set to validate the trained tea garden drought prediction model. If the validation fails, the training set is expanded to continue training. If the validation passes, then:
[0146] The ROC curve and the calibration curve are drawn by the test set to test the tea garden drought prediction model that has passed the verification. If the test fails, the training set is expanded to continue training; if the test passes, the training is terminated.
[0147] The data set is divided into layers at a ratio of 8:1:1 to preserve the sample distribution characteristics; overfitting is prevented by setting dual conditions of loss function convergence and early stopping; MAE (mean absolute error), MSE (mean square error), and RMSE (root mean square error) are used to evaluate the accuracy of numerical predictions in the verification phase; ROC curve (discrimination) and calibration curve (probability calibration) are used for dual verification in the testing phase to ensure the reliability of the model in actual scenarios; and the training set is dynamically expanded when verification / testing fails to form a "training-feedback" closed loop to continuously optimize the model's generalization ability.
[0148] The ROC curve is used to evaluate the discriminative ability of the tea plantation drought prediction model. The area under the ROC curve (AUC) measures the model's efficiency. An ideal model's ROC curve is as close to the upper left corner as possible, indicating a high TPR and a low FPR, demonstrating excellent discriminative ability. The AUC quantifies the model's overall discriminative ability. In statistical analysis, an AUC value greater than 0.7 is considered to indicate excellent discriminative performance. Higher AUC values indicate a stronger model's ability to distinguish between positive and negative classes.
[0149] A calibration curve is a method used to assess the accuracy of a disease risk model in predicting the probability of future occurrence. In the present invention, a calibration curve is used to assess the degree of fit between the drought level changes in a tea garden under different key factor parameters and the actual values.
[0150] In summary, the advantages of the present invention are:
[0151] 1. A large amount of historical monitoring data is collected from the tea garden through the sensor array to construct a data set and perform sample expansion operations. Then, the data set is analyzed by the Limma algorithm, COX survival regression model, GLM generalized linear model and SHAP to obtain the environmental factor correlation graph; then, a tea garden drought prediction model is created based on the sequentially connected multimodal feature extraction layer, dual-branch prediction layer and fusion output layer, and the loss function of the tea garden drought prediction model is set; the multimodal feature extraction layer is used to extract the core feature group from the input detection data through the environmental factor correlation graph; the dual-branch prediction layer is used to perform nonlinear modeling and high-dimensional linear classification on the core feature group through the LightGBM branch and the LSVM branch, and output the tree enhanced probability prediction value and the kernel support decision value; the fusion output layer is used to fuse the tree enhanced probability prediction value and the kernel support decision value, and output the tea garden drought prediction result; then the data set is divided into training set, validation set and test set, the tea garden drought prediction model is trained by the training set and the loss function, the trained tea garden drought prediction model is verified by the validation set, and the validation set is verified by the test set. The tea garden drought prediction model that has passed the test is tested and deployed. Finally, real-time monitoring data is collected from the tea garden through a sensor array. After preprocessing, the real-time monitoring data is input into the tea garden drought prediction model to obtain the tea garden drought prediction results, which are then displayed on a visualization interface built based on the Shiny and DynNom frameworks. In other words, the tea garden drought prediction model is used to predict tea garden drought. The tea garden drought prediction model combines LightGBM and LSVM, effectively integrating the advantages of the two. Compared with SVM, LSVM has the advantages of short training time, low computational complexity, suitability for large-scale data sets, and strong generalization ability. The fusion output layer is used to fuse the tree enhancement probability prediction value and the kernel support decision value through a dynamic weighting strategy, replacing the traditional simple feature splicing. The environmental factor correlation map is combined to select and adjust the weight of each feature, that is, the weights of the features corresponding to the N environmental factors that have the greatest impact on drought prediction under the current time segment label are increased, ultimately greatly improving the timeliness, accuracy, and generalization ability of tea garden drought prediction.
[0152] 2. By adopting an array of 13 types of sensors (covering multiple dimensions such as meteorology, soil, and environment), more than 40 environmental parameters are collected to ensure that the data covers all elements of the tea garden ecosystem and avoid single-dimensional bias; by combining K-nearest neighbor filling (preserving local features) and Z-score+K-means outlier repair (dynamic clustering to correct global distribution), data quality is improved, which is better than traditional mean filling or simple filtering methods; by introducing time segment labels (months) and spatial location labels, and expanding the sample through the autoregressive model, the sample imbalance problem caused by seasonal fluctuations in agricultural data is effectively solved.
[0153] 3. The Limma algorithm (biostatistics) is used to screen significant factors, and the COX survival model (risk analysis) and GLM (generalized linear regression) are used to calculate the risk level. Combined with SHAP (explainable AI) to quantify the contribution, dual verification of statistical models and machine learning is achieved, so that the selected environmental factors have both statistical significance and practical explanatory power, avoiding overfitting or spurious correlation interference.
[0154] 4. Through the standardized processing of the spatiotemporal coding module and the generation of spatiotemporal cross-features (such as the combination of "daytime humidity-nighttime temperature difference"), the dynamic evolution of the tea garden microenvironment can be effectively captured; through the feature screening module, directional filtering is performed based on the environmental factor correlation map to reduce redundant feature interference.
[0155] 5. Through the LightGBM branch based on the leaf-wise growth strategy, feature interactions (such as the nonlinear relationship between "soil conductivity and rainfall") are efficiently mined and probability predictions are output; through the RBF kernel function of the LSVM branch, high-dimensional space is mapped to solve complex boundary classification problems (such as fuzzy scenarios of drought level thresholds), and decision confidence is output. The dynamic weighted fusion strategy is used to adaptively adjust the weights of the two branches (such as LSVM confidence is mainly used in the early stage of drought, and LightGBM probability is mainly used in the persistent period), effectively improving the robustness of the prediction.
[0156] 6. The sample distribution characteristics are retained by dividing the data set into layers at a ratio of 8:1:1; overfitting is prevented by setting dual conditions of loss function convergence and early stopping; MAE (mean absolute error), MSE (mean square error), and RMSE (root mean square error) are used to evaluate the accuracy of numerical predictions in the verification phase; ROC curve (discrimination) and calibration curve (probability calibration) are used for dual verification in the testing phase to ensure the reliability of the model in actual scenarios; the training set is dynamically expanded when verification / testing fails, forming a "training-feedback" closed loop to continuously optimize the model's generalization ability.
[0157] 7. The dynamic visualization interface built through the Shiny+DynNom framework supports multi-dimensional display of drought level, spatial location, and confidence level, making it easier for agricultural managers to quickly locate high-risk areas and formulate irrigation strategies.
[0158] 8. By innovatively applying the Limma algorithm used for differential gene expression in biostatistics to environmental factor screening, we can identify environmental factors that change significantly under time segment labels through linear model fitting and Bayesian correction, thereby improving the statistical rigor of feature screening and avoiding the misjudgment of time series non-independent data by traditional analysis of variance (ANOVA).
[0159] 9. To address seasonal fluctuations in agricultural data (such as the imbalance in data volume between the rainy season and the dry season), an autoregressive model (AR) is used to generate synthetic data that conforms to the laws of time series, rather than simple oversampling, to avoid introducing noise and enhance the model's ability to predict rare climate events (such as extreme droughts).
[0160] 10. By combining month (time) and sampling location (space) labels, we can expand the data in a targeted manner, for example, simulating the changes in soil parameters at the same location in different months, and enhancing the model's adaptability to spatiotemporal heterogeneity.
[0161] 11. By integrating multi-dimensional sensor data with interdisciplinary analysis methods (Limma algorithm, COX model, GLM and SHAP), and combining the advantages of LightGBM and LSVM dual-branch models, a tea garden drought prediction framework with high precision and strong interpretability was constructed; by adopting spatiotemporal coding and dynamic feature screening mechanisms to accurately extract core environmental factors, and optimizing data quality through autoregressive data enhancement and Z-score+K-means anomaly repair, the dynamic weighted fusion of dual models was used to improve the robustness of prediction, and the spatiotemporal visualization of drought was achieved based on Shiny+DynNom, ultimately forming a closed-loop system from data collection, model training to real-time early warning, which significantly solved the problems of data sparsity, model generalization and decision credibility in agricultural scenarios, and provided efficient and scalable technical support for intelligent drought management in tea gardens.
[0162] Although the specific embodiments of the present invention are described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included within the scope of protection of the claims of the present invention.
Claims
1. A tea garden drought prediction method integrating LightGBM and LSVM, characterized by: The steps include: Step S1, collecting a large amount of historical monitoring data from the tea garden through a sensor array, constructing a data set after preprocessing each of the historical monitoring data, and performing a sample expansion operation on the data set; Step S2, analyzing the data set using the Limma algorithm, COX survival regression model, GLM generalized linear model, and SHAP to obtain an environmental factor correlation graph; Step S3, creating a tea garden drought prediction model based on the sequentially connected multimodal feature extraction layer, the dual-branch prediction layer, and the fusion output layer, and setting a loss function of the tea garden drought prediction model; The multimodal feature extraction layer is used to extract the core feature group from the input detection data through the environmental factor correlation graph; the dual-branch prediction layer is used to perform nonlinear modeling and high-dimensional linear classification on the core feature group through the LightGBM branch and the LSVM branch, and output the tree enhancement probability prediction value and the kernel support decision value; the fusion output layer is used to fuse the tree enhancement probability prediction value and the kernel support decision value, and output the tea garden drought prediction result; Step S4, dividing the data set into a training set, a validation set, and a test set, training the tea garden drought prediction model using the training set and the loss function, validating the trained tea garden drought prediction model using the validation set, testing the verified tea garden drought prediction model using the test set, and deploying the tea garden drought prediction model that has passed the test; Step S5: Real-time monitoring data is collected from the tea garden through a sensor array. After preprocessing the real-time monitoring data, the data is input into the deployed tea garden drought prediction model to obtain a tea garden drought prediction result, and the tea garden drought prediction result is displayed on a visualization interface built based on the Shiny and DynNom framework.
2. The tea garden drought prediction method integrating LightGBM and LSVM as claimed in claim 1, characterized in that: The step S1 is specifically as follows: A large amount of historical monitoring data is collected from the tea garden through a sensor array including humidity sensors, light intensity sensors, temperature sensors, carbon dioxide sensors, air pressure sensors, particulate matter sensors, global solar radiation sensors, pH sensors, salinity sensors, conductivity sensors, rainfall sensors, wind speed and direction sensors, and positioners; The historical monitoring data at least include maximum light intensity, average light intensity, sunshine duration, maximum temperature, minimum temperature, average temperature, average daytime temperature, average nighttime temperature, daily temperature difference, maximum air humidity, minimum air humidity, average relative humidity, average daytime air humidity, average nighttime air humidity, maximum carbon dioxide concentration, minimum carbon dioxide concentration, average carbon dioxide concentration, average daytime carbon dioxide concentration, average nighttime carbon dioxide concentration, maximum air pressure, minimum air pressure, average air pressure, average daytime air pressure, average nighttime air pressure, air pressure difference, daily average PM2.5, daily average PM10, maximum TBQ global radiation, minimum TBQ global radiation, daily average TBQ global radiation, maximum soil temperature, minimum soil temperature, daily average soil temperature, soil temperature difference, maximum soil pH, minimum soil pH, average soil pH, maximum soil salinity, minimum soil salinity, average soil salinity, maximum soil electrical conductivity, minimum soil electrical conductivity, average soil electrical conductivity, rainfall, wind direction, wind speed, sampling location and sampling time; After filling missing values and repairing outliers for each of the historical monitoring data, each of the historical monitoring data is labeled with soil entropy, drought level, and time segment labels to complete the preprocessing of each of the historical monitoring data. A data set is constructed based on the preprocessed historical monitoring data, and a sample expansion operation is performed on the data set through an autoregressive model; the time segment label is the month of the sampling time; The missing value filling is based on the K-nearest neighbor filling method; the outlier repair is based on the Z-score algorithm and the K-means clustering algorithm, specifically: Perform Z-score standardization on each of the historical monitoring data, calculate the Z-score value of each of the historical monitoring data after Z-score standardization, and compare the Z-score value with a preset abnormality threshold to screen out abnormal values; cluster the historical monitoring data after Z-score standardization using a K-means clustering algorithm to obtain several cluster centers, and repair each abnormal value in turn based on the nearest cluster center.
3. The tea garden drought prediction method integrating LightGBM and LSVM as claimed in claim 1, characterized in that: The step S2 is specifically as follows: Segmenting the data set based on time segmentation labels to obtain a number of data subsets, and screening significant factors between the data subsets whose change rates exceed a preset change threshold using the Limma algorithm; Calculating the first drought risk level of each of the significant factors using a COX survival regression model, calculating the second drought risk level of each of the significant factors using a GLM generalized linear model, and cross-validating the first drought risk level and the second drought risk level to screen N environmental factors with the greatest impact on drought conditions under different time segment labels from each of the significant factors, where N is a positive integer; The contribution of each of the screened environmental factors is calculated by SHAP, and an environmental factor correlation graph is constructed based on the screened environmental factors, contribution and time segment labels.
4. The tea garden drought prediction method integrating LightGBM and LSVM as claimed in claim 1, characterized in that: In step S3, the multimodal feature extraction layer is composed of a spatiotemporal encoding module and a feature screening module; the spatiotemporal encoding module is used to perform spatiotemporal encoding on the input monitoring data after standardization processing to obtain spatiotemporal cross features; the feature screening module is used to screen the core feature group from each spatiotemporal cross feature through the environmental factor correlation diagram; The dual-branch prediction layer is composed of a LightGBM branch and an LSVM branch; the LightGBM branch is used to mine feature interactions in the core feature group through a leaf-wise growth strategy to perform nonlinear modeling and output a tree-enhanced probability prediction value representing the probability of drought occurrence; The LSVM branch is used to process the nonlinear boundary of the core feature group through the RBF kernel to perform high-dimensional linear classification, and output a kernel support decision value representing the classification confidence; The fusion output layer is used to fuse the tree enhanced probability prediction value and the kernel support decision value through a dynamic weighting strategy, and output the tea garden drought prediction result containing the drought period, drought level, drought location and confidence.
5. The tea garden drought prediction method integrating LightGBM and LSVM as claimed in claim 1, characterized in that: The step S4 is specifically as follows: Using a stratified sampling technique, the dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:
1. The tea garden drought prediction model is trained using the training set. During the training process, the hyperparameters of the tea garden drought prediction model, including at least a learning rate, a batch size, and a number of training rounds, are continuously optimized until the loss function converges or a preset early stopping condition is met; The MAE, MSE and RMSE are calculated using the validation set to validate the trained tea garden drought prediction model. If the validation fails, the training set is expanded to continue training. If the validation passes, then: The ROC curve and the calibration curve are drawn by the test set to test the tea garden drought prediction model that has passed the verification. If the test fails, the training set is expanded to continue training; if the test passes, the training is terminated.
6. A tea garden drought prediction system integrating LightGBM and LSVM, characterized by: Includes the following modules: A data set construction module is used to collect a large amount of historical monitoring data from the tea garden through a sensor array, construct a data set after pre-processing each of the historical monitoring data, and perform a sample expansion operation on the data set; An environmental factor correlation graph generation module is used to analyze the data set using the Limma algorithm, the COX survival regression model, the GLM generalized linear model, and SHAP to obtain an environmental factor correlation graph; A tea garden drought prediction model creation module is used to create a tea garden drought prediction model based on the sequentially connected multimodal feature extraction layer, the dual-branch prediction layer, and the fusion output layer, and set the loss function of the tea garden drought prediction model; The multimodal feature extraction layer is used to extract the core feature group from the input detection data through the environmental factor correlation graph; the dual-branch prediction layer is used to perform nonlinear modeling and high-dimensional linear classification on the core feature group through the LightGBM branch and the LSVM branch, and output the tree enhancement probability prediction value and the kernel support decision value; the fusion output layer is used to fuse the tree enhancement probability prediction value and the kernel support decision value, and output the tea garden drought prediction result; a tea garden drought prediction model training module, configured to divide the data set into a training set, a validation set, and a test set; train the tea garden drought prediction model using the training set and a loss function; verify the trained tea garden drought prediction model using the validation set; test the verified tea garden drought prediction model using the test set; and deploy the tested tea garden drought prediction model; The tea garden drought prediction module is used to collect real-time monitoring data from the tea garden through a sensor array, pre-process the real-time monitoring data, input the deployed tea garden drought prediction model to obtain the tea garden drought prediction results, and display the tea garden drought prediction results on a visualization interface built based on the Shiny and DynNom frameworks.
7. The tea garden drought prediction system integrating LightGBM and LSVM as claimed in claim 6, characterized in that: The dataset construction module is specifically used for: A large amount of historical monitoring data is collected from the tea garden through a sensor array including humidity sensors, light intensity sensors, temperature sensors, carbon dioxide sensors, air pressure sensors, particulate matter sensors, global solar radiation sensors, pH sensors, salinity sensors, conductivity sensors, rainfall sensors, wind speed and direction sensors, and positioners; The historical monitoring data at least include maximum light intensity, average light intensity, sunshine duration, maximum temperature, minimum temperature, average temperature, average daytime temperature, average nighttime temperature, daily temperature difference, maximum air humidity, minimum air humidity, average relative humidity, average daytime air humidity, average nighttime air humidity, maximum carbon dioxide concentration, minimum carbon dioxide concentration, average carbon dioxide concentration, average daytime carbon dioxide concentration, average nighttime carbon dioxide concentration, maximum air pressure, minimum air pressure, average air pressure, average daytime air pressure, average nighttime air pressure, air pressure difference, daily average PM2.5, daily average PM10, maximum TBQ global radiation, minimum TBQ global radiation, daily average TBQ global radiation, maximum soil temperature, minimum soil temperature, daily average soil temperature, soil temperature difference, maximum soil pH, minimum soil pH, average soil pH, maximum soil salinity, minimum soil salinity, average soil salinity, maximum soil electrical conductivity, minimum soil electrical conductivity, average soil electrical conductivity, rainfall, wind direction, wind speed, sampling location and sampling time; After filling missing values and repairing outliers for each of the historical monitoring data, each of the historical monitoring data is labeled with soil entropy, drought level, and time segment labels to complete the preprocessing of each of the historical monitoring data. A data set is constructed based on the preprocessed historical monitoring data, and a sample expansion operation is performed on the data set through an autoregressive model; the time segment label is the month of the sampling time; The missing value filling is based on the K-nearest neighbor filling method; the outlier repair is based on the Z-score algorithm and the K-means clustering algorithm, specifically: Perform Z-score standardization on each of the historical monitoring data, calculate the Z-score value of each of the historical monitoring data after Z-score standardization, and compare the Z-score value with a preset abnormality threshold to screen out abnormal values; cluster the historical monitoring data after Z-score standardization using a K-means clustering algorithm to obtain several cluster centers, and repair each abnormal value in turn based on the nearest cluster center.
8. The tea garden drought prediction system integrating LightGBM and LSVM as claimed in claim 6, characterized in that: The environmental factor correlation graph generation module is specifically used for: Segmenting the data set based on time segmentation labels to obtain a number of data subsets, and screening significant factors between the data subsets whose change rates exceed a preset change threshold using the Limma algorithm; Calculating the first drought risk level of each of the significant factors using a COX survival regression model, calculating the second drought risk level of each of the significant factors using a GLM generalized linear model, and cross-validating the first drought risk level and the second drought risk level to screen N environmental factors with the greatest impact on drought conditions under different time segment labels from each of the significant factors, where N is a positive integer; The contribution of each of the screened environmental factors is calculated by SHAP, and an environmental factor correlation graph is constructed based on the screened environmental factors, contribution and time segment labels.
9. The tea garden drought prediction system integrating LightGBM and LSVM according to claim 6, characterized in that: In the tea garden drought prediction model creation module, the multimodal feature extraction layer is composed of a spatiotemporal encoding module and a feature screening module; the spatiotemporal encoding module is used to perform spatiotemporal encoding on the input monitoring data after standardization processing to obtain spatiotemporal cross features; the feature screening module is used to screen the core feature group from each spatiotemporal cross feature through the environmental factor correlation diagram; The dual-branch prediction layer is composed of a LightGBM branch and an LSVM branch; the LightGBM branch is used to mine feature interactions in the core feature group through a leaf-wise growth strategy to perform nonlinear modeling and output a tree-enhanced probability prediction value representing the probability of drought occurrence; The LSVM branch is used to process the nonlinear boundary of the core feature group through the RBF kernel to perform high-dimensional linear classification, and output a kernel support decision value representing the classification confidence; The fusion output layer is used to fuse the tree enhanced probability prediction value and the kernel support decision value through a dynamic weighting strategy, and output the tea garden drought prediction result containing the drought period, drought level, drought location and confidence.
10. The tea garden drought prediction system integrating LightGBM and LSVM according to claim 6, characterized in that: The tea garden drought prediction model training module is specifically used for: Using a stratified sampling technique, the dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:
1. The tea garden drought prediction model is trained using the training set. During the training process, the hyperparameters of the tea garden drought prediction model, including at least a learning rate, a batch size, and a number of training rounds, are continuously optimized until the loss function converges or a preset early stopping condition is met; The MAE, MSE and RMSE are calculated using the validation set to validate the trained tea garden drought prediction model. If the validation fails, the training set is expanded to continue training. If the validation passes, then: The ROC curve and the calibration curve are drawn by the test set to test the tea garden drought prediction model that has passed the verification. If the test fails, the training set is expanded to continue training; if the test passes, the training is terminated.
Citation Information
Cited By
Regional drought monitoring and evaluation method and system based on big data
CN122087360A
Big data-based regional drought monitoring and evaluation method and system
CN122087360B