An ensemble learning-based system for predicting crop yields

Abstract

An ensemble learning-based system (100) for predicting crop yields, wherein the system (100) comprises the following: a data acquisition unit (102) configured to acquire data sets that include information on crop production, environmental characteristics and soil fertilization; a preprocessing unit (104) connected to the acquisition unit (102) and configured for processing missing values ​​of the data set using multiple imputation by chained equation (MICE) and normalizing the data using robust scaling (RS) to handle outliers; a classification unit (106) with holdout (70%-30%), 5-fold cross-validation (5-fold CV) and 10-fold cross-validation, which is connected to the preprocessing unit (104) and is configured to split the preprocessed data into training and test datasets; an ensemble-based forecasting unit (108) comprising a bagging subunit (108a), a boosting subunit (108b) and a stacking subunit (108c), which is connected to the classification unit (106) and is configured to forecast crop yield; and an evaluation unit (110) configured to evaluate the accuracy and F1 score of the ensemble-based prediction unit (108).

Description

FIELD OF THE INVENTION

[0001] The present invention relates to the field of precision agriculture. In particular, the present invention relates to the field of crop yield prediction. More specifically, the present invention relates to an ensemble learning-based crop yield prediction system. BACKGROUND OF THE INVENTION

[0002] The agricultural sector plays a critical role in feeding the world's population, particularly in populous countries like India, where unpredictable weather patterns and climate change pose a significant threat to food security. Farmers frequently face challenges such as droughts and floods, which impact crop yields and productivity. Given the increase in global population and worsening climatic conditions, there is an urgent need for improvements in agricultural practices to meet the growing demand for food. Research shows that agricultural production must be significantly improved to feed a growing global population and cope with environmental changes and resource scarcity.

[0003] Crop yield prediction (CYP) has emerged as a key task in agriculture, providing valuable insights for crop management, resource allocation, and government decision-making regarding food distribution, imports, and exports. Accurate forecast models can help optimize resource utilization and support efficient agricultural practices. However, predicting crop yields is a complex task, as it involves numerous variables such as weather conditions, soil properties, and crop-specific factors, all of which interact in complex ways.

[0004] Conventional machine learning (ML) techniques have been used to address the CYP problem, but they often struggle with handling missing data, outliers, and the large number of predictor variables involved. Such data irregularities, arising from varying input attributes and long-term data collection, can degrade the performance of traditional ML classifiers.

[0005] To overcome these challenges, ensemble learning (EL) techniques have been developed as an advanced approach to improve prediction accuracy. EL methods combine multiple machine learning models or base learners to improve performance by reducing overfitting, managing data inconsistencies, and providing robust predictions. In recent years, various EL methods, including bagging, boosting, and stacking generalization (SG), have been applied in agricultural research to improve crop yield prediction models.

[0006] Despite advances in the application of EL techniques for crop yield prediction, no comprehensive comparative analysis of these approaches has yet been published. Researchers have implemented specialized EL methods such as Gradient Boosting (GB), Random Forest (RF), and XGBoost to predict crop yields in various agricultural environments. However, there is still a need for a thorough evaluation of the most effective EL techniques for crop yield prediction, especially in scenarios where data preprocessing is required to handle missing values ​​and outliers.

[0007] To overcome these limitations, an ensemble learning-based crop yield prediction system must be developed. The system performs a comparative analysis of different EL techniques, both with and without data preprocessing, to determine the optimal approach for accurate crop yield prediction. The system leverages several EL methods, including bagging, boosting, and SG, applied to preprocessed and unpreprocessed datasets. By incorporating advanced preprocessing techniques such as Multiple Imputation by Chained Equation (MICE) to handle missing values ​​and Robust Scaling (RS) for outlier management, the system aims to provide a comprehensive solution for improving the accuracy and reliability of crop yield predictions.

[0008] The technical advances disclosed by the present invention overcome the limitations and disadvantages of existing and conventional systems and methods. SUMMARY OF THE INVENTION

[0009] The present invention relates to a crop yield prediction (CYP) system using ensemble learning (EL) techniques to address the growing demand for food in emerging economies due to rising populations. The system is designed to predict crop yields with high accuracy by processing complex, nonlinear factors such as environmental conditions, soil quality, fertilizers, and crop production data. The data is obtained from the International Crops Research Institute for the Semi-Arid Tropics (ICRISAT) for 33 districts of Assam over a period of 28 years. The system uses a combination of bagging, boosting, and stacking generalization (SG) techniques to achieve the best performance.

[0010] An object of the present invention is to provide an ensemble learning based crop yield prediction system.

[0011] Another objective of the present invention is to improve prediction accuracy by using data preprocessing techniques such as MICE data imputation to handle missing values ​​and RS normalization to make the dataset robust to outliers.

[0012] Another objective of the present invention is to analyze and compare the performance of different bagging techniques such as Extra Trees (ET), Random Forest (RF) and Canonical Bagging (CB), where ET shows the best performance in terms of accuracy (85.79%) and f-1 score (84.97%) using 10-fold cross-validation.

[0013] Another objective of the present invention is to demonstrate that the bagging ensemble approach performs better than boosting and SG for CYP when applied with data preprocessing, while the boosting ensemble approach performs better without data preprocessing.

[0014] Another objective of the present invention is to propose future improvements by incorporating feature selection and feature extraction to further improve the performance of EL techniques for CYP.

[0015] Another objective of the present invention is to conclude that ensemble learning with data preprocessing provides better results for CYP compared to approaches without preprocessing, thus highlighting the importance of preprocessing in precision agriculture models.

[0016] The present invention aims to provide an ensemble learning-based crop yield prediction system, the system comprising: an acquisition unit configured to acquire datasets containing information on crop yield, environmental properties, and soil fertilizer; a preprocessing unit connected to the acquisition unit and configured to process missing values ​​of the dataset using multiple imputation by chained equation (MICE) and to normalize the data using robust scaling (RS) to remove outliers; a classification unit with holdout (70%-30%), 5-fold cross-validation (5-fold CV), and 10-fold cross-validation, connected to the preprocessing unit and configured to split the preprocessed data into training and test datasets;an ensemble-based prediction unit comprising a bagging subunit, a boosting subunit, and a stacking subunit, connected to the classification unit and configured to predict crop yields; and an evaluation unit configured to evaluate the accuracy and F1 score of the ensemble-based prediction unit.

[0017] To further clarify the advantages and features of the present disclosure, a more detailed description of the invention will be given with reference to specific embodiments thereof, which are illustrated in the accompanying drawings. It should be noted that these drawings represent only typical embodiments of the invention and are therefore not to be considered limiting its scope. The invention will be described and explained in additional detail and in greater detail with reference to the accompanying drawings. SHORT DESCRIPTION OF THE FIGURE

[0018] These and other features, aspects, and advantages of the present disclosure will be better understood when the following detailed description is read with reference to the accompanying drawings, in which like characters represent like parts throughout. Where: Fig. 1 shows a block diagram of an ensemble learning-based crop yield prediction system according to an embodiment of the present disclosure.

[0019] Furthermore, those skilled in the art will appreciate that elements in the drawings are shown for convenience and may not necessarily be drawn to scale. For example, the flowcharts illustrate the method by key steps to enhance understanding of aspects of the present disclosure. Furthermore, with respect to device construction, one or more components of the device may have been represented in the drawings by conventional symbols, and the drawing may show only the specific details relevant to understanding embodiments of the present disclosure in order not to clutter the drawings with details that would be readily apparent to those skilled in the art who would benefit from the description herein. DETAILED DESCRIPTION:

[0020] To facilitate an understanding of the principles of the invention, reference will now be made to the embodiment illustrated in the drawings and described in specific language. It is to be understood, however, that no limitation on the scope of the invention is intended thereby, since such changes and further modifications of the illustrated system, and such further applications of the principles of the invention as illustrated therein, are contemplated as would normally occur to one skilled in the art to which the invention pertains.

[0021] It will be understood by one skilled in the art that the foregoing general description and the following detailed description are exemplary and explanatory of the invention and are not intended to be limiting thereof.

[0022] References in this specification to "one aspect," "another aspect," or similar language mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Therefore, occurrences of the phrase "in one embodiment," "in another embodiment," and similar language throughout this specification may or may not all refer to the same embodiment.

[0023] The terms "comprises," "comprising," or other variations thereof are intended to cover non-exclusive inclusion, such that a process or method comprising a list of steps not only includes those steps, but may also include other steps not expressly listed or inherent in such process or method. Likewise, one or more devices or subsystems or elements or structures or components preceded by "comprises..." does not preclude, without further limitation, the existence of other devices or other subsystems or other elements or other structures or other components or additional devices or additional subsystems or additional elements or additional structures or additional components.

[0024] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The system, methods, and examples provided herein are for illustrative purposes only and should not be considered limiting.

[0025] Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0026] Fig.1 shows a block diagram of an ensemble learning-based system (100) for predicting crop yields, the system (100) comprising: an acquisition unit (102) configured to acquire data sets containing information on crop production, environmental properties, and soil fertilizer; a preprocessing unit (104) connected to the acquisition unit (102) and configured to process missing values ​​of the data set using multiple imputation by chained equation (MICE) and to normalize the data using robust scaling (RS) to remove outliers; a classification unit (106) with holdout (70%-30%), 5-fold cross-validation (5-fold CV), and 10-fold cross-validation, connected to the preprocessing unit (104) and configured to split the preprocessed data into training and test data sets;an ensemble-based prediction unit (108) comprising a bagging subunit (108a), a boosting subunit (108b), and a stacking subunit (108c), connected to the classification unit (106) and configured to predict crop yield; and an evaluation unit (110) configured to assess the accuracy and F1 score of the ensemble-based prediction unit (108);

[0027] In one embodiment, the dataset comprises 123 attributes, including 12 monthly characteristics of actual evapotranspiration, 12 monthly characteristics of potential evapotranspiration, 12 monthly characteristics of climatic water deficit, 12 monthly characteristics of precipitation, 12 monthly characteristics of maximum temperature, 12 monthly characteristics of minimum temperature, 12 monthly characteristics of wind speed, 12 monthly characteristics of runoff, 12 monthly characteristics of precipitation, crop area, crop volume, crop yield, 4 characteristics of nitrogen use, 4 characteristics of phosphate use, 4 characteristics of potash use, and an outcome attribute representing crop type, with 23 different crop categories.

[0028] In one embodiment, the MICE in the preprocessing unit (104) predicts, estimates, and treats missing values ​​to reduce model inaccuracies caused by incomplete data.

[0029] In one embodiment, the RS in the preprocessing unit (104) handles outliers by scaling attributes based on the interquartile range, thus ensuring robust data normalization.

[0030] In one embodiment, the bagging subunit (108a) in the ensemble-based prediction unit (108) uses the Extra Trees (ET) classifier as the primary classifier for superior performance in predicting crop yields for 23 crop categories after 10-fold cross-validation.

[0031] In one embodiment, the ET classifier with 10-fold cross-validation shows the highest accuracy of 85.79% and an F1 score of 84.97%.

[0032] In one embodiment, the boosting subunit (108b) in the ensemble-based prediction unit (108) performs iterative model updates, focusing on hard-to-classify data using classifiers such as XGBoost, AdaBoost, and GB to improve performance in scenarios without data preprocessing.

[0033] In one embodiment, the stacking subunit (108c) in the ensemble-based prediction unit (108) comprises a first layer combining classifiers selected from Extra Trees, XGBoost, LightGBM, Random Forest, and Support Vector Machines, and a second layer comprising a meta-learner selected from XGBoost or SVM to optimize the final predictions.

[0034] The proposed system uses advanced machine learning to predict crop yields based on a large-scale dataset. This dataset comprises 123 attributes collected over a period of 28 years from the 33 districts of Assam. The attributes encompass a wide range of parameters related to agriculture, the environment, and soil fertility, such as monthly data on evapotranspiration (both actual and potential), climatic water deficit, rainfall, maximum and minimum temperatures, wind speed, runoff, rainfall amount, and crop-specific production data. Additionally, data on fertilizer use (nitrogen, phosphate, and potash) are recorded, and the outcome variable is the type of crop grown, with a total of 23 different crop categories.

[0035] To address challenges such as missing data and outliers, this system integrates robust preprocessing techniques. Missing values, which can affect the predictive power of machine learning algorithms, are handled using Multiple Imputation by Chained Equations (MICE). This technique provides a reliable way to predict and account for the uncertainty associated with missing data, thus improving the integrity of the dataset. To manage outliers, this system applies Robust Scaling (RS), which scales data based on the interquartile range, ensuring that outliers do not distort the model's predictions.

[0036] For classification, the system uses three approaches: a holdout method with a 70% to 30% split between training and testing, 5-fold cross-validation, and 10-fold cross-validation. These techniques ensure that the model's performance is tested on different data samples, reducing the likelihood of overfitting and ensuring generalizability across different data subsets.

[0037] Ensemble learning (EL) techniques form the core of the predictive mechanism of the present system, and a comparative analysis of bagging, boosting, and stacking techniques was conducted. Bagging, known to reduce variance and avoid overfitting, is implemented through techniques such as canonical bagging (CB), random forest (RF), and extra trees (ET), which improve model stability. Boosting, which improves model performance by iteratively adjusting the weights of hard-to-classify samples, is applied with AdaBoost, gradient boosting (GB), and XGBoost. These boosting techniques refine the model's accuracy by focusing on more difficult-to-classify data, thus improving the model's predictive power.

[0038] Stacked Generalization (SG), a fusion learning technique, is also used in the present system to further improve prediction performance. Three stacked architectures are implemented: Stacked_1 uses Extra Trees and XGBoost as the base learner with a Support Vector Machine (SVM) as the meta-learner; Stacked_2 uses LightGBM, RF, and GB as the base learner with ET as the meta-learner; and Stacked_3 integrates RF, ET, SVM, LightGBM, and GB as the base learner with XGBoost as the meta-learner in the second layer. These stacked models combine the strengths of multiple base learners, thus improving both accuracy and generalization.

[0039] In one embodiment, the present system evaluates these ensemble techniques using accuracy and F1 score as key performance metrics, ensuring that the model not only performs well in terms of precision but also achieves balanced predictions across all crop categories. The use of multiple ensemble learning approaches and comprehensive performance evaluation make the system a robust tool for crop yield prediction and support better decision-making in agricultural management and planning.

[0040] In one embodiment, the proposed system is experimentally validated. The main objective of the present system is to predict crop yield using the district-level dataset covering crop production data and various environmental and soil properties over 28 years. This dataset required extensive data preprocessing because it contained missing values ​​and outliers, which were treated using the Multiple Imputation by Chained Equations (MICE) technique for imputation and Robust Scaling (RS) to handle outliers. The preprocessed dataset was then used to train several Ensemble Learning (EL) classifiers using three different training methods: holdout (70%-30% split), 5-fold cross-validation (5-fold CV), and 10-fold cross-validation (10-fold CV). The performance of the proposed system in terms of accuracy measurement parameters is listed below in Table 1: Model Techniques Holdout (70%-30%) 5-fold CV 10-fold CV sagging Canonical Bagging 66.58 % 74.60 % 84.40 % Random Forest 67.2 % 74.77 % 84.58 % Additional tree 67.22 % 74.87 % 85.79 % Reinforcement AdaBoost 67.22 % 74.86 % 84.58 % Gradient amplification 66.30 % 73.88 % 84.57 % XGBoost 67.22 % 74.87 % 84.56 % Stacking Generalization (SG) Stacked_1 53.58 % 67.58 % 77.61 % Stacked_2 48.26 % 64.59 % 75.39 % Stacked_3 42.76 % 66.25 % 78.60 %

[0041] The performance of the present system is evaluated using two key performance metrics: accuracy and F1 score. The results showed that the Bagging Ensemble approach, specifically the Extra Trees (ET) classifier, provided the best predictive performance for crop yield classification among the 23 crop categories. When trained with the 10-fold cross-validation method, the Extra Trees classifier achieved the highest accuracy of 85.79%, outperforming other bagging techniques such as Canonical Bagging (CB) and Random Forest (RF). In terms of F1 score, the Extra Trees classifier again outperformed the other techniques, achieving the highest F1 score of 84.97%, demonstrating its robustness in correctly classifying crop yields. The performance of the proposed system with respect to the F1 score measurement parameters is shown below in Table II: Model Techniques Holdout (70%-30%) 5-fold lifetime on 10-fold CV sagging Canonical Bagging 66.81 % 74.42 % 84.30 % Random Forest 66.85 % 73.97 % 84.45 % Additional tree 67.54 % 74.40 % 84.97 % Reinforcement AdaBoost 67.51 % 74.10 % 83.77 % Gradient amplification 67.21 % 72.86 % 83.01 % XGBoost 67.54 % 74.85 % 84.58 % Stacking Generalization (SG) Stacked_1 53.89 % 65.73 % 76.94 % Stacked_2 48.18 % 65.02 % 75.09 % Stacked_3 42.58 % 66.22 % 78.54 %

[0042] Comparative analysis of different EL techniques, including bagging, boosting, and stacked generalization (SG), further supported that the bagging approach, especially the Extra Trees classifier, is superior in both accuracy and F1 score. Experiments also highlighted the effectiveness of the 10-fold cross-validation method, which consistently yielded better results than the holdout and 5-fold CV training methods. This demonstrates the importance of using a rigorous cross-validation technique to improve the generalization and reliability of crop yield predictions in diverse agricultural environments. The proposed system therefore provides an effective and reliable solution for crop yield prediction using advanced ensemble learning methods and thorough data preprocessing techniques.

[0043] In one embodiment, the SG ensemble approach does not work without data normalization because the SG ensemble approach is unable to handle outliers. Therefore, in this experiment, only bagging and boosting approaches are performed without data preprocessing. The performance of the EL classifiers without data preprocessing with respect to the accuracy measurement parameter is listed below in Table III: Model Techniques Holdout (70%-30%) 5-fold CV 10-fold CV sagging Canonical Bagging 58.94 % 71.09 % 80.10 % Random Forest 59.28 % 68.51 % 82.53 % Additional tree 61.70 % 71.18 % 82.63 % Reinforcement AdaBoost 61.89 % 72.79 % 81.56 % Gradient boost 60.56 % 70.58 % 80.89 % XGBoost 62.90 % 72.77 % 82.78 %

[0044] In one embodiment, the boosting ensemble approach provides the best accuracy for crop yield prediction (CYP) compared to bagging and stacked generalization (SG) approaches without data preprocessing. Among the boosting techniques employed, XGBoost, which uses a 10-fold cross-validation (CV) training method, provides the highest accuracy at 82.78%, outperforming other boosting techniques such as AdaBoost and Gradient Boosting (GB) in classifying 23 different crop categories.

[0045] Similarly, as shown in Table IV below, the performance of the ensemble classifiers is measured using the F1 score as a key metric. Here, too, the boosting approach proves superior to bagging and SG without data preprocessing. XGBoost achieves the best result in terms of F1 score under 10-fold CV with a value of 82.70%, demonstrating its effectiveness in balancing precision and recall for accurate crop classification. Other boosting methods such as AdaBoost and GB show comparatively lower performance: Model Techniques Holdout (70%-30%) 5-fold CV 10-fold CV sagging Canonical Bagging 59.80 % 69.62 % 81.05 % Random Forest 59.53 % 68.71 % 82.60 % Additionally the tree 62.56 % 71.44 % 82.66 % Reinforcement AdaBoost 59.73 % 72.39 % 82.54 % Gradient gain 60.26 % 69.72 % 80.56 % XGBoost 60.19 % 71.35 % 82.70 %

[0046] The experiments demonstrate that the performance of ensemble learning classifiers is significantly improved by proper data preprocessing, including handling missing values ​​and outliers. Without preprocessing, boosting, especially XGBoost, still delivers respectable results, but its accuracy and F1 score lag behind models trained with preprocessed data. When comparing the ensemble methods boosting, bagging, and SG, SG performs below average due to the complexity of combining multiple baseline models into an optimal predictor. The SG approach struggles to identify the most effective technique for the meta-learner, and its performance varies depending on the domain and the problem at hand.This indicates that while SG has potential, its dependence on different baseline models and the difficulty of selecting the best meta-learner limits its effectiveness for the CYP problem.

[0047] The drawings and the foregoing description provide examples of embodiments. Those skilled in the art will recognize that one or more of the described elements may well be combined into a single functional element. Alternatively, certain elements may be separated into multiple functional elements. Elements from one embodiment may be added to another embodiment. For example, orders of the processes described herein may be changed and are not limited to the manner described herein. Furthermore, the actions of any flowchart need not be implemented in the order shown; nor do all actions necessarily need to be performed. Also, those actions that are not dependent on other actions may be performed in parallel with the other actions. The scope of the embodiments is in no way limited by these specific examples.Numerous variations, whether explicitly stated in the specification or not, such as differences in structure, dimensions, and material use, are possible. The scope of the embodiments is at least as broad as indicated in the following claims.

[0048] Advantages, other benefits, and solutions to problems have been described above with reference to specific embodiments. However, the advantages, benefits, solutions to problems, and any components that may cause an advantage or solution to occur or become more apparent are not to be construed as a critical, required, or essential feature or component of any or all of the claims. REFERENCES 100 An ensemble learning-based system for crop yield prediction. 102 collection unit 104 Preprocessing unit 106 Classification Unit 108 Ensemble-Based Forecast Unit 108a Bagging sub-unit 108b Boosting subunit 108c stacking subunit 110 Evaluation unit

Claims

[1] An ensemble learning-based system (100) for predicting crop yields, the system (100) comprising: a recording unit (102) configured to record data sets including information on crop production, environmental properties and soil fertilization; a preprocessing unit (104) connected to the acquisition unit (102) and configured to process missing values ​​of the data set using multiple imputation by chained equation (MICE) and to normalize the data using robust scaling (RS) to handle outliers; a classification unit (106) with holdout (70%-30%), 5-fold cross-validation (5-fold CV) and 10-fold cross-validation, connected to the preprocessing unit (104) and configured to split the preprocessed data into training and test data sets; an ensemble-based prediction unit (108) comprising a bagging subunit (108a), a boosting subunit (108b), and a stacking subunit (108c), which is connected to the classification unit (106) and is configured to predict the crop yield; and an evaluation unit (110) configured to evaluate the accuracy and the F1 score of the ensemble-based prediction unit (108). [2] The system (100) of claim 1, wherein the dataset comprises 123 attributes, including 12 monthly characteristics of actual evapotranspiration, 12 monthly characteristics of potential evapotranspiration, 12 monthly characteristics of climatic water deficit, 12 monthly characteristics of precipitation, 12 monthly characteristics of maximum temperature, 12 monthly characteristics of minimum temperature, 12 monthly characteristics of wind speed, 12 monthly characteristics of runoff, 12 monthly characteristics of precipitation, crop area, crop volume, crop yield, 4 characteristics of nitrogen use, 4 characteristics of phosphate use, 4 characteristics of potash use, and a result attribute representing crop type, with 23 different crop categories. [3] The system (100) of claim 1, wherein the MICE in the preprocessing unit (104) predicts, estimates, and treats missing values ​​to reduce model inaccuracies caused by incomplete data. [4] The system (100) of claim 1, wherein the RS in the preprocessing unit (104) handles outliers by scaling attributes based on the interquartile range, thereby ensuring robust data normalization. [5] The system (100) of claim 1, wherein the bagging subunit (108a) in the ensemble-based prediction unit (108) uses the Extra Trees (ET) classifier as the primary classifier for superior performance in predicting crop yield for 23 crop categories after 10-fold cross-validation. [6] The system (100) of claim 5, wherein the ET classifier with 10-fold cross-validation has the highest accuracy of 85.79% and an F1 score of 84.97%. [7] The system (100) of claim 1, wherein the boosting subunit (108b) in the ensemble-based prediction unit (108) performs iterative model updates, focusing on hard-to-classify data and using classifiers such as XGBoost, AdaBoost, and GB to improve performance in scenarios without data preprocessing. [8] The system (100) of claim 1, wherein the stacking subunit (108c) in the ensemble-based prediction unit (108) comprises a first layer combining classifiers selected from Extra Trees, XGBoost, LightGBM, Random Forest, and Support Vector Machines, and a second layer comprising a meta-learner selected from XGBoost or SVM to optimize the final predictions.

Images