Grain depot grain condition state prediction method and system based on error-driven ensemble learning
By using an error-driven ensemble learning framework, the bias of weak classifiers towards the majority class is dynamically corrected, improving the accuracy and stability of grain condition monitoring, solving the prediction problem under extremely imbalanced data, and achieving early warning and reliable prediction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SINOGRAIN CHENGDU STORAGE RESEARCH INSTITUTE CO LTD
- Filing Date
- 2026-03-24
- Publication Date
- 2026-05-29
AI Technical Summary
The existing grain condition monitoring system suffers from extremely imbalanced data, resulting in poor accuracy in identifying abnormal grain conditions. Traditional methods are difficult to adapt to different grain types, storage types, and regions, and the models have low prediction accuracy and recall for a few categories of samples.
An error-driven ensemble learning approach is adopted. The dataset is divided into multiple training subsets by hierarchical sampling. Weak classifiers are trained and misclassified samples are identified to form enhanced training subsets. The weak classifiers are optimized through multiple rounds of iteration. Finally, the grain condition status ensemble classifier is integrated by probability averaging.
It significantly improves the recall rate and prediction accuracy for abnormal states, enhances the stability and generalization ability of the classifier, adapts to different grain storage conditions, realizes early prediction and reliable early warning, and reduces sensitivity to noise and local fluctuations.
Smart Images
Figure CN121920934B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of grain detection technology, specifically to a method and system for predicting grain condition in grain depots based on error-driven ensemble learning. Background Technology
[0002] During grain storage, abnormal conditions such as heating and condensation may occur inside the grain pile due to respiration, microbial activity, and environmental changes. If these conditions are not detected and addressed in a timely manner, they can lead to grain mold, quality deterioration, and even significant economic losses. Therefore, real-time and accurate monitoring of grain conditions is crucial.
[0003] Currently, the analysis and assessment of grain conditions mainly rely on real-time data collected by sensor networks deployed within grain warehouses. Traditional methods commonly employ rule-based judgments based on fixed thresholds, i.e., pre-setting empirical thresholds for temperature, humidity, or their gradients; when monitored data exceeds these thresholds, it is considered abnormal. While this method is simple to implement, its thresholds heavily depend on expert experience, making it difficult to apply universally to different grain types, warehouse types, regions, and storage stages. Furthermore, it is prone to numerous misjudgments due to environmental fluctuations or sensor noise.
[0004] With the development of machine learning technology, data-driven prediction models are increasingly being applied in this field. For example, models such as random forests and gradient boosting decision trees (e.g., XGBoost) are used to learn the complex mapping relationship between states and features from historical data. While these methods offer improved pattern recognition capabilities compared to thresholding methods, in practical applications, long-term monitoring data often exhibits extreme class imbalances in grain condition data. Normal state samples dominate, while samples of feverish states are scarce, and samples of condensation states, representing high-risk mold conditions, are extremely rare. This imbalanced data distribution leads to the model over-optimizing its fit to the majority class (normal state) during training, severely lacking the ability to learn feature patterns of the minority class (abnormal states). Ultimately, this results in low prediction accuracy and recall for feverish and condensation states, failing to meet the requirements for high-reliability predictions.
[0005] To address the imbalance of state samples, common techniques include adjustments at the data and algorithm levels. At the data level, oversampling techniques (such as SMOTE) or undersampling techniques (such as RUS) are typically used to artificially balance the data distribution. However, these methods are prone to introducing synthetic noise or losing important sample information, potentially impairing the model's generalization ability. At the algorithm level, adjustments are made to the loss function or sample weights to increase attention to samples from minority classes. However, these methods fail to fully consider the differences in learning difficulty between different classes, and even within the same class of samples, in grain condition data. The model struggles to dynamically and accurately focus on those difficult-to-predict samples with blurred boundaries to normal states or atypical features, resulting in poor accuracy in predicting abnormal states. Summary of the Invention
[0006] This invention aims to solve the technical problem of poor accuracy in identifying abnormal grain conditions under extremely unbalanced data in existing grain condition monitoring, and proposes a grain depot grain condition prediction method and system based on error-driven ensemble learning.
[0007] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:
[0008] In a first aspect, the present invention provides a method for predicting grain condition status in grain depots based on error-driven ensemble learning, the method comprising:
[0009] Step 1: Collect grain condition monitoring data from multiple monitoring points in the grain depot in chronological order, and determine the grain condition status at each monitoring moment. The grain condition monitoring data includes temperature and humidity data, and the grain condition status includes normal status, heat generation status, and condensation status.
[0010] Step 2: Based on the grain condition monitoring data and corresponding grain condition status at each monitoring time, construct the original training dataset, in which the number of samples in the normal state is greater than the number of samples in the fever state and the number of samples in the condensation state.
[0011] Step 3: Use stratified sampling to divide the original training dataset into multiple training subsets;
[0012] Step 4: Perform the following iterative optimization process:
[0013] Step 41: Based on each training subset, train a weak classifier for predicting the grain condition status.
[0014] Step 42: Apply each weak classifier to the remaining training subsets other than its own corresponding training subset to predict the grain condition status, identify the misjudged samples that were incorrectly predicted by the weak classifier, and construct the hard sample set corresponding to each weak classifier based on the misjudged samples.
[0015] Step 43: For each weak classifier, merge the hard sample set into the corresponding training subset to form an enhanced training subset. Based on the enhanced training subset, update the corresponding weak classifier.
[0016] Step 44: Repeat steps 42 to 43 for multiple iterations until the preset iteration termination condition is met;
[0017] Step 5: Integrate all the iteratively optimized weak classifiers to form the final grain condition status integrated classifier. Based on the real-time grain condition monitoring data and the grain condition status integrated classifier, predict the real-time grain condition status.
[0018] Furthermore, in step 3, the stratified sampling technique specifically includes:
[0019] Step 31: Set the original training dataset The total number of samples is The number of samples in the normal state was [number missing]. The number of samples in the fever state is The number of samples in the condensation state is And satisfy ;
[0020] Step 32: Through stratified sampling, the original training dataset is... Divided into training subset During sampling, each training subset The target sample sizes for the normal state, fever state, and condensation state are respectively: ;
[0021] Step 33: Evaluate the representativeness of the sampling results based on the hypergeometric distribution and Chebyshev's inequality. If the representativeness condition is met, the sampling is complete; otherwise, re-sampling is performed. The representativeness condition is:
[0022] ;
[0023] in, This represents the index of grain condition status. Represents the training subset The Middle The sample proportion bias of grain planting conditions , Represents the training subset The Middle The sample proportion of grain planting conditions Represents the original training dataset The Middle The sample proportion of grain planting conditions This indicates the preset deviation threshold. This represents the probability that the absolute value of the sample proportion deviation is greater than or equal to the deviation threshold δ. , indicating a training subset The Middle The variance of the sample size for the grain planting condition status. Represents the original training dataset The Middle The number of samples for the status of grain planting.
[0024] Furthermore, in step 42, for the first... A weak classifier The corresponding hard sample set is formally defined as:
[0025] ;
[0026] in, Indicates the first The set of hard samples corresponding to each weak classifier This represents a feature vector constructed based on grain condition monitoring data. This indicates the true state of grain supply. Indicates the first A weak classifier for the feature vector Predicted grain conditions This indicates the difference between the predicted and actual grain condition. Inconsistent Indicates misjudged samples. This is a set difference operation, representing the original training dataset. Remove the first training subset All remaining samples.
[0027] Furthermore, the feature vector includes the temperature gradient at different grain pile depths, the dew point temperature difference between the grain layer and the environment, and its rate of change over time.
[0028] Furthermore, in step 5, all the iteratively optimized weak classifiers are integrated, specifically including:
[0029] By integrating weak classifiers using a probability averaging method, a grain condition status integrated classifier is obtained. The parameters of the grain condition status integrated classifier are the mean values of the parameters of each weak classifier.
[0030] ;
[0031] in, The parameters represent the integrated classifier for grain condition status. Indicates the first The parameters of a weak classifier This indicates the number of training subsets, i.e., the number of weak classifiers.
[0032] Further, in step 44, the iteration termination condition is:
[0033] In two consecutive iterations, the weighted F1 scores of all weak classifiers for heating and condensation states improved by less than the set threshold, or reached the preset maximum number of iterations.
[0034] Furthermore, the method also includes:
[0035] When constructing the original grain condition training dataset, a portion of the samples are reserved to form an independent test set; the generalization performance of the obtained grain condition ensemble classifier is verified on the independent test set.
[0036] The generalization performance of the grain condition status integrated classifier satisfies the following constraints:
[0037] Based on the McDiarmid inequality, for a preset error fluctuation threshold The empirical error calculated by the grain condition status integrated classifier on the independent test set. Its mathematical expectation The deviation between them exceeds probability satisfy:
[0038] ;
[0039] in, Indicates the number of samples in the independent test set. Represents the natural exponential function;
[0040] Based on Rademacher complexity, the true error of the grain condition integrated classifier satisfy:
[0041] ;
[0042] in, This represents the dimension of the feature vector input to the grain condition ensemble classifier. For asymptotic upper bounds, This indicates that it includes all growth rates not exceeding [a certain percentage]. A set of functions that are constant multiples of each other.
[0043] Furthermore, in step 5, the grain condition status integrated classifier outputs the probability that the real-time grain condition monitoring data belongs to the normal state, the overheating state, or the condensation state. When the probability of the overheating state or the condensation state exceeds the warning threshold, an abnormal grain condition warning is triggered.
[0044] Furthermore, the base model of the weak classifier is any one of decision tree, random forest, XGBoost, or LightGBM.
[0045] Secondly, the present invention provides a grain depot grain condition prediction system based on error-driven ensemble learning, used to implement the grain depot grain condition prediction method based on error-driven ensemble learning as described in the first aspect, the system comprising:
[0046] The data acquisition module is used to collect grain condition monitoring data from multiple monitoring points in the grain depot in a time sequence and determine the grain condition status at each monitoring moment. The grain condition monitoring data includes temperature and humidity data, and the grain condition status includes normal status, heat generation status, and condensation status.
[0047] The dataset construction module is used to construct the original training dataset based on the grain condition monitoring data and its corresponding grain condition status at each monitoring time. The number of samples in the normal state is greater than the number of samples in the fever state and the number of samples in the condensation state.
[0048] The dataset partitioning module is used to divide the original training dataset into multiple training subsets using stratified sampling techniques.
[0049] The iterative optimization module is used to perform the following iterative optimization process:
[0050] For each training subset, a weak classifier is trained to predict the grain condition status.
[0051] Each weak classifier is applied to the remaining training subsets other than its own corresponding training subset to predict the grain condition status, identify the misjudged samples that were incorrectly predicted by the weak classifier, and construct the hard sample set corresponding to each weak classifier based on the misjudged samples.
[0052] For each weak classifier, the set of hard samples is merged into the corresponding training subset to form an enhanced training subset. Based on the enhanced training subset, the corresponding weak classifier is updated.
[0053] Repeat the above steps of predicting grain condition and updating the weak classifier, and iterate for multiple rounds until the preset iteration termination condition is met.
[0054] The grain condition prediction module is used to integrate all the iteratively optimized weak classifiers to form the final grain condition integrated classifier, and to predict the real-time grain condition based on the real-time grain condition monitoring data and the grain condition integrated classifier.
[0055] The beneficial effects of this invention are as follows: The grain condition prediction method and system based on error-driven ensemble learning provided by this invention, through an error-driven mechanism, dynamically and accurately directs the resources of weak classifiers to the weakest link in historical predictions, effectively correcting the inherent bias of weak classifiers towards the majority class (normal state), thereby significantly improving the recall rate and prediction accuracy for a few abnormal states. Furthermore, the method of training and ensembled using multiple independent training subsets enhances the overall stability and generalization ability of the classifier, reducing its sensitivity to noisy data and local fluctuations. This method does not rely on manually setting fixed thresholds or complex cost matrices, exhibits strong adaptability, and can adapt to the data characteristics of different grain depots under different grain storage conditions, providing grain depots with earlier and more reliable grain condition prediction capabilities, realizing the transformation from post-event alarms to pre-event predictions, and gaining valuable time for taking preventative measures and ensuring grain storage safety. Attached Figure Description
[0056] Figure 1A flowchart illustrating the grain condition prediction method based on error-driven ensemble learning provided for an embodiment;
[0057] Figure 2 A schematic diagram of the iterative optimization process provided for the embodiments;
[0058] Figure 3 This is a schematic diagram of the structure of a grain depot grain condition prediction system based on error-driven ensemble learning, provided as an example. Detailed Implementation
[0059] Existing traditional machine learning models for predicting grain conditions struggle to effectively overcome the low accuracy and recall caused by extreme data imbalance, and fail to fully consider the differences in learning difficulty within samples of grain condition data. Therefore, this invention proposes a technical solution.
[0060] In this invention, stratified sampling is first used to create multiple representative training subsets with consistent distributions based on the original training dataset, which suffers from extreme class imbalance. This lays the foundation for subsequent differential learning. Subsequently, weak classifiers trained on each training subset are used to cross-predict other training subsets, systematically identifying samples misclassified by the weak classifiers (i.e., difficult samples). These misclassified samples typically belong to minority classes that are difficult to learn or have ambiguous boundaries. The identified difficult samples are then merged into their corresponding training subsets, forcing the weak classifiers to focus more on these weaknesses in the next round of learning. Through multiple iterations, the decision boundaries of the weak classifiers are continuously refined. Finally, by integrating all these iteratively optimized weak classifiers, a powerful grain condition status ensemble classifier is formed. This invention solves the problem of predicting abnormal states caused by extreme sample imbalance by enabling weak classifiers to actively learn from their historical prediction errors, achieving a directed, data-driven self-evolution.
[0061] The technical solutions in this embodiment will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0062] Figure 1 A flowchart illustrating a grain depot condition prediction method based on error-driven ensemble learning is shown. Please refer to [link / reference]. Figure 1 The method includes the following steps:
[0063] Step 1: Collect grain condition monitoring data from multiple monitoring points in the grain depot in chronological order, and determine the grain condition status at each monitoring time. The grain condition monitoring data includes temperature and humidity data, and the grain condition status includes normal status, heat generation status, and condensation status.
[0064] In practical applications, the first step is to collect grain condition monitoring data through a sensor network pre-deployed within the grain depot (such as flat warehouses or silos). Sensors are typically arranged in a three-dimensional grid at different levels of the grain pile, including the upper, middle, and lower sections, as well as key locations such as the edges and center, to comprehensively perceive the internal environment of the grain pile. The core data collected are time-series data of temperature and humidity, collected continuously in time series or at a fixed frequency (e.g., hourly), forming a sequence reflecting the changes in the physical state of each monitoring point over time.
[0065] Then, based on professional knowledge of grain storage, historical experience thresholds, manual verification and annotation by grain condition experts, and alarm records, the grain condition at each monitoring moment was divided into three categories: normal state, overheating state, and condensation state. Among them, the normal state represents stable grain condition; the overheating state represents abnormal temperature rise caused by pests, microbial activity, etc.; and the condensation state represents a high-risk state where moisture condenses inside the grain pile due to temperature differences, which can easily lead to mold growth.
[0066] Step 2: Based on the grain condition monitoring data and corresponding grain condition status at each monitoring time, construct the original training dataset, in which the number of samples in the normal state is greater than the number of samples in the fever state and the number of samples in the condensation state.
[0067] Specifically, the grain condition monitoring data at each monitoring time and each valid monitoring point is treated as an independent feature sample, and the grain condition status determined at that time is used as the true label corresponding to that feature sample, thus forming a labeled sample. Collecting all such samples generated at all historical times constitutes the original training dataset. Over a long period, the original training dataset exhibits an extremely imbalanced distribution: the number of samples in normal condition is far greater than those in overheated condition, and the number of samples in overheated condition is greater than the extremely rare samples in dewy condition.
[0068] Step 3: Use stratified sampling to divide the original training dataset into multiple training subsets.
[0069] In this embodiment, the stratified sampling technique specifically includes steps 31 to 33:
[0070] Step 31: Set the original training dataset The total number of samples is The number of samples in the normal state was The number of samples in the fever state is The number of samples in the condensation state is And satisfy ;
[0071] Step 32: Through stratified sampling, the original training dataset is... Divided into training subset During sampling, each training subset The target sample sizes for the normal state, fever state, and condensation state are respectively: ;
[0072] Step 33: Evaluate the representativeness of the sampling results based on the hypergeometric distribution and Chebyshev's inequality. If the representativeness condition is met, the sampling is complete; otherwise, resampling is performed. The representativeness condition is:
[0073] ;
[0074] in, This represents the index of grain condition status. Represents the training subset The Middle The sample proportion bias of grain planting conditions , Represents the training subset The Middle The sample proportion of grain planting conditions Represents the original training dataset The Middle The sample proportion of grain planting conditions This indicates the preset deviation threshold. This represents the probability that the absolute value of the sample proportion deviation is greater than or equal to the deviation threshold δ. , indicating a training subset The Middle The variance of the sample size for the grain planting condition status. Represents the original training dataset The Middle The number of samples for the status of grain planting.
[0075] Specifically, the purpose of stratified sampling is to make each training subset Internally maintains the same data as the original training dataset The category distribution (proportion of normal, feverish, and condensation states) is consistent, ensuring that each training subset can be considered a representative independent observation from the original imbalanced distribution. Ideally, each training subset... The sample sizes for normal state, fever state, and condensation state should be respectively... However, since it may not be divisible in actual operation, it is not possible to directly sample according to this sample size. Based on this, this embodiment uses the above ideal situation as the target sample size for approximate stratified sampling.
[0076] To theoretically ensure the representativeness of the sampling results, this embodiment also uses statistical analysis based on the hypergeometric distribution and Chebyshev inequality to theoretically evaluate the sampling results to ensure their representativeness. The evaluation criterion is: for any training subset... and any grain situation status ( (corresponding to normal, fever, and condensation respectively), corresponding to the sample proportion deviation. The probability that the absolute value exceeds the preset deviation threshold δ It must not be greater than its theoretical upper limit. This representativeness condition strictly constrains the range of sampling bias from a probabilistic perspective. When this representativeness condition is met, the sampling result is considered representative, and the partitioning is complete; otherwise, resampling is required until the representativeness condition is met.
[0077] The above steps, by introducing rigorous statistical evaluation, ensure the scientific validity and reliability of the training subset partitioning, providing a prerequisite for subsequent error-driven iteration and ensemble learning to effectively handle imbalanced data and improve the prediction performance of heating and condensation states.
[0078] Step 4: Perform the following iterative optimization process:
[0079] Step 41: Based on each training subset, train a weak classifier for predicting the grain condition status.
[0080] Step 42: Apply each weak classifier to the remaining training subsets other than its own corresponding training subset to predict the grain condition status, identify the misjudged samples that were incorrectly predicted by the weak classifier, and construct the hard sample set corresponding to each weak classifier based on the misjudged samples.
[0081] Step 43: For each weak classifier, merge the hard sample set into the corresponding training subset to form an enhanced training subset. Based on the enhanced training subset, update the corresponding weak classifier.
[0082] Step 44: Repeat steps 42 to 43 for multiple iterations until the preset iteration termination condition is met.
[0083] Please see Figure 2 Specifically, this step uses a dynamic self-correcting iterative cycle to make the weak classifier continuously focus on grain condition samples that are difficult to predict accurately, especially samples of heat and condensation that are misjudged, thereby refining the decision boundary.
[0084] In practical applications, firstly in each training subset Train an initial weak classifier independently. The weak classifiers learn the basic mapping relationship between grain condition status (normal, feverish, condensation) and the feature vectors corresponding to grain condition monitoring data from their respective training subsets. The base model for the weak classifiers can be any one of decision trees, random forests, XGBoost, or LightGBM. Because each training subset maintains the overall representativeness of the original data distribution through stratified sampling, the patterns learned by the weak classifiers have both commonalities and diversity due to subtle differences in the data subsets.
[0085] The feature vector is constructed based on grain condition monitoring data, including the temperature gradient at different grain pile depths, the dew point temperature difference between the grain layer and the environment, and its rate of change over time.
[0086] Then, for each weak classifier, it is applied to all sample data except for its own training subset for prediction. Through this cross-validation method, misclassified samples that are incorrectly predicted by the weak classifier are systematically identified. These misclassified samples constitute the hard sample set corresponding to the weak classifier, representing the weak classifier's insufficient learning of key abnormal states.
[0087] For the A weak classifier The corresponding hard sample set is formally defined as:
[0088] ;
[0089] in, Indicates the first The set of hard samples corresponding to each weak classifier This represents a feature vector constructed based on grain condition monitoring data. This indicates the true state of grain supply. Indicates the first A weak classifier for the feature vector Predicted grain conditions This indicates the difference between the predicted and actual grain condition. Inconsistent Indicates misjudged samples. This is a set difference operation, representing the original training dataset. Remove the first training subset All remaining samples.
[0090] Next, the set of difficult samples is merged into the training subset corresponding to its weak classifier, thus forming a new augmented training subset. This increases the relative weight of difficult samples in the training data, especially minority class anomalous state samples. Subsequently, based on the augmented training subset, the weak classifier is updated or retrained, forcing it to focus on samples that were not properly handled in the previous round in the next learning round, thereby selectively correcting its decision boundary.
[0091] Finally, multiple iterations are performed following the above steps until a preset iteration termination condition is met. In this embodiment, the iteration termination condition is: in two consecutive iterations, the weighted F1 score improvement of all weak classifiers for heating and condensation states is less than a set threshold, or the preset maximum number of iterations is reached. Through multiple iterations, the weak classifiers can continuously and dynamically focus on the difficulties in historical predictions, allowing the performance of a series of weak classifiers to continuously improve and differentiate during iteration.
[0092] Step 5: Integrate all the iteratively optimized weak classifiers to form the final grain condition status integrated classifier. Based on the real-time grain condition monitoring data and the grain condition status integrated classifier, predict the real-time grain condition status.
[0093] In this embodiment, all iteratively optimized weak classifiers are integrated, specifically including:
[0094] By integrating weak classifiers using a probability averaging method, a grain condition status integrated classifier is obtained. The parameters of the grain condition status integrated classifier are the mean values of the parameters of each weak classifier.
[0095] ;
[0096] in, The parameters represent the integrated classifier for grain condition status. Indicates the first The parameters of a weak classifier This indicates the number of training subsets, i.e., the number of weak classifiers.
[0097] This step is used to integrate all the iteratively optimized weak classifiers into a more robust and stable final prediction model, namely the grain condition ensemble classifier.
[0098] In practical applications, this embodiment employs a probability averaging method to calculate the average parameters of all weak classifiers, thereby integrating the knowledge learned by each weak classifier from different perspectives and training subsets. During the prediction process, for newly input real-time grain condition monitoring data, a corresponding feature vector is constructed using the same method as during training. This vector is then input into the grain condition status ensemble classifier to obtain the probability distribution of the real-time grain condition monitoring data belonging to three states: normal, overheated, and condensation. Finally, the grain condition state with the highest probability value is selected as the judgment result. When the probability of the overheated or condensation state exceeds a pre-set warning threshold, a corresponding graded warning signal is triggered.
[0099] By employing the aforementioned probability averaging ensemble strategy, the risk of overfitting that may exist with a single classifier can be effectively reduced, and the generalization ability and prediction robustness of the classifier can be significantly improved when faced with grain condition data noise, local fluctuations, and imbalanced sample distribution.
[0100] In this embodiment, the method further includes:
[0101] When constructing the original grain condition training dataset, a portion of the samples are reserved to form an independent test set; the generalization performance of the obtained grain condition ensemble classifier is verified on the independent test set.
[0102] The generalization performance of the grain condition status integrated classifier satisfies the following constraints:
[0103] Based on the McDiarmid inequality, for a preset error fluctuation threshold The empirical error calculated by the grain condition status integrated classifier on the independent test set. Its mathematical expectation The deviation between them exceeds probability satisfy:
[0104] ;
[0105] in, Indicates the number of samples in the independent test set. Represents the natural exponential function;
[0106] Based on Rademacher complexity, the true error of the grain condition integrated classifier satisfy:
[0107] ;
[0108] in, This represents the dimension of the feature vector input to the grain condition ensemble classifier. For asymptotic upper bounds, This indicates that it includes all growth rates not exceeding [a certain percentage]. A set of functions that are constant multiples of each other.
[0109] This step is used to theoretically verify the final integrated classifier of grain condition status to ensure the reliability and interpretability of its performance.
[0110] In practical applications, firstly, when constructing the original training dataset in step 2, a certain proportion (such as 70%-30%) of the partitioning strategy is adopted to reserve a portion of the labeled samples that do not participate in any training and iteration process, forming an independent test set.
[0111] The resulting grain condition status ensemble classifier is then evaluated on an independent test set. Its performance is measured not only by conventional metrics (such as precision and recall) but also, theoretically, by satisfying the following key constraints:
[0112] 1. Error stability guarantee (based on McDiarmid's inequality):
[0113] The empirical error calculated by the grain condition ensemble classifier on the independent test set The observed average error rate is highly stable, even for any preset threshold of small error fluctuations. empirical error Deviating from its theoretical expected value Exceed The probability satisfies the following constraints: As long as the number of samples in the independent test set used for validation is sufficient, the observed classifier performance (such as 95% accuracy) is highly unlikely to be accidental, but rather a stable reflection of its true ability, thus guaranteeing the credibility of the performance evaluation results in terms of probability.
[0114] 2. Guarantee of upper bound on generalization error (based on Rademacher complexity):
[0115] This constraint is used to ensure the true error of the integrated grain condition classifier. Able to be influenced by empirical errors with a high probability The term is defined by adding a convergence term, i.e. This theoretically guarantees the effectiveness and robustness of the method.
[0116] The above steps, by reserving an independent test set and combining the core inequalities in the above statistical learning theory, not only provide an empirical evaluation of the model performance, but also mathematically demonstrate the generalization ability and stability of the weak classifier obtained by the error-driven ensemble learning method, significantly enhancing the theoretical rigor and reliability of the entire technical solution.
[0117] In summary, the grain condition prediction method based on error-driven ensemble learning provided in this embodiment addresses the challenge of predicting key abnormal states such as overheating and condensation caused by extreme class imbalances in grain condition monitoring. Through an error-driven ensemble learning framework, it dynamically mines and strengthens the weak classifier's learning of historical misjudged samples (difficult samples), effectively correcting the weak classifier's inherent bias towards the majority class (normal state). The combination of stratified sampling and theoretical evaluation ensures the representativeness of data partitioning. Through multiple rounds of iterative optimization and probabilistic averaging ensemble, the final constructed grain condition ensemble classifier significantly improves the recall rate for condensation and overheating states on multiple real grain depot datasets, while maintaining a controllable overall false alarm rate. Furthermore, theoretical analysis (McDiarmid inequality and Rademacher complexity) provides rigorous guarantees for its generalization performance and stability, realizing a shift from passive threshold alarms to early and accurate prediction, and providing reliable technical support for proactive grain depot security and loss reduction. Compared to methods such as random undersampling, oversampling, and static adjustment of loss functions or sample weights, this embodiment does not statically adjust the data distribution. Instead, it dynamically identifies difficult samples that are misclassified by the classifier and guides a series of weak classifiers to continuously focus on these difficult-to-learn samples through multiple rounds of iterative enhancement, thereby continuously refining the decision boundary. This process makes the final integrated strong classifier more sensitive to minority class patterns.
[0118] Based on the above technical solution, this embodiment also provides a grain depot grain condition prediction system based on error-driven ensemble learning. Please refer to [link to relevant documentation]. Figure 3 The system is used to implement the grain depot grain condition prediction method based on error-driven ensemble learning as described in the embodiments, and the system includes:
[0119] The data acquisition module is used to collect grain condition monitoring data from multiple monitoring points in the grain depot in a time sequence and determine the grain condition status at each monitoring moment. The grain condition monitoring data includes temperature and humidity data, and the grain condition status includes normal status, heat generation status, and condensation status.
[0120] The dataset construction module is used to construct the original training dataset based on the grain condition monitoring data and its corresponding grain condition status at each monitoring time. The number of samples in the normal state is greater than the number of samples in the fever state and the number of samples in the condensation state.
[0121] The dataset partitioning module is used to divide the original training dataset into multiple training subsets using stratified sampling techniques.
[0122] The iterative optimization module is used to perform the following iterative optimization process:
[0123] For each training subset, a weak classifier is trained to predict the grain condition status.
[0124] Each weak classifier is applied to the remaining training subsets other than its own corresponding training subset to predict the grain condition status, identify the misjudged samples that were incorrectly predicted by the weak classifier, and construct the hard sample set corresponding to each weak classifier based on the misjudged samples.
[0125] For each weak classifier, the set of hard samples is merged into the corresponding training subset to form an enhanced training subset. Based on the enhanced training subset, the corresponding weak classifier is updated.
[0126] Repeat the above steps of predicting grain condition and updating the weak classifier, and iterate for multiple rounds until the preset iteration termination condition is met.
[0127] The grain condition prediction module is used to integrate all the iteratively optimized weak classifiers to form the final grain condition integrated classifier, and to predict the real-time grain condition based on the real-time grain condition monitoring data and the grain condition integrated classifier.
[0128] It is understood that the grain depot grain condition prediction system based on error-driven ensemble learning described in this embodiment is a system used to implement the grain depot grain condition prediction method based on error-driven ensemble learning described in the embodiment. As the system disclosed in the embodiment corresponds to the method disclosed in the embodiment, the description is relatively simple. For relevant parts, please refer to the description of the method. It will not be repeated here.
Claims
1. A method for predicting grain condition status in grain depots based on error-driven ensemble learning, characterized in that, The method includes: Step 1: Collect grain condition monitoring data from multiple monitoring points in the grain depot in chronological order, and determine the grain condition status at each monitoring time. The grain condition monitoring data includes temperature and humidity data, and the grain condition status includes normal status, heat generation status, and condensation status. Step 2: Based on the grain condition monitoring data and corresponding grain condition status at each monitoring time, construct the original training dataset, in which the number of samples in the normal state is greater than the number of samples in the fever state and the number of samples in the condensation state. Step 3: Use stratified sampling to divide the original training dataset into multiple training subsets; Step 4: Perform the following iterative optimization process: Step 41: Based on each training subset, train a weak classifier for predicting the grain condition status. Step 42: Apply each weak classifier to the remaining training subsets other than its own corresponding training subset to predict the grain condition status, identify the misjudged samples that were incorrectly predicted by the weak classifier, and construct the hard sample set corresponding to each weak classifier based on the misjudged samples. Step 43: For each weak classifier, merge the hard sample set into the corresponding training subset to form an enhanced training subset. Based on the enhanced training subset, update the corresponding weak classifier. Step 44: Repeat steps 42 to 43 for multiple iterations until the preset iteration termination condition is met; Step 5: Integrate all the iteratively optimized weak classifiers to form the final grain condition status integrated classifier. Based on the real-time grain condition monitoring data and the grain condition status integrated classifier, predict the real-time grain condition status.
2. The grain depot condition prediction method based on error-driven ensemble learning according to claim 1, characterized in that, In step 3, the stratified sampling technique specifically includes: Step 31: Set the original training dataset The total number of samples is The number of samples in the normal state was [number missing]. The number of samples in the fever state is The number of samples in the condensation state is And satisfy ; Step 32: Through stratified sampling, the original training dataset is... Divided into training subset Each training subset during sampling The target sample sizes for the normal state, fever state, and condensation state are respectively: ; Step 33: Evaluate the representativeness of the sampling results based on the hypergeometric distribution and Chebyshev's inequality. If the representativeness condition is met, the sampling is complete; otherwise, re-sampling is performed. The representativeness condition is: ； in, This represents the index of grain condition status. Represents the training subset The Middle The sample proportion bias of grain planting conditions , Represents the training subset The Middle The sample proportion of grain planting conditions Represents the original training dataset The Middle The sample proportion of grain planting conditions This indicates the preset deviation threshold. This represents the probability that the absolute value of the sample proportion deviation is greater than or equal to the deviation threshold δ. , indicating a training subset The Middle The variance of the sample size for the grain planting condition status. Represents the original training dataset The Middle The number of samples for the status of grain planting.
3. The grain depot grain condition prediction method based on error-driven ensemble learning according to claim 1, characterized in that, In step 42, for the first A weak classifier The corresponding hard sample set is formally defined as: ； in, Indicates the first The set of hard samples corresponding to each weak classifier This represents a feature vector constructed based on grain condition monitoring data. This indicates the true state of grain supply. Indicates the first A weak classifier for the feature vector Predicted grain conditions This indicates the difference between the predicted and actual grain condition. Inconsistent Indicates misjudged samples. For set difference operations, represents the original training dataset. Remove the first training subset All remaining samples.
4. The grain depot grain condition prediction method based on error-driven ensemble learning according to claim 3, characterized in that, The feature vector includes the temperature gradient at different grain pile depths, the dew point temperature difference between the grain layer and the environment, and its rate of change over time.
5. The grain depot grain condition prediction method based on error-driven ensemble learning according to claim 1, characterized in that, Step 5 integrates all the iteratively optimized weak classifiers, specifically including: By integrating weak classifiers using a probability averaging method, a grain condition status integrated classifier is obtained. The parameters of the grain condition status integrated classifier are the mean values of the parameters of each weak classifier. ； in, The parameters represent the integrated classifier for grain condition status. Indicates the first The parameters of a weak classifier This indicates the number of training subsets, i.e., the number of weak classifiers.
6. The grain depot grain condition prediction method based on error-driven ensemble learning according to claim 1, characterized in that, In step 44, the iteration termination condition is: In two consecutive iterations, the weighted F1 scores of all weak classifiers for heating and condensation states improved by less than the set threshold, or reached the preset maximum number of iterations.
7. The grain depot grain condition prediction method based on error-driven ensemble learning according to claim 1, characterized in that, The method further includes: When constructing the original grain condition training dataset, a portion of the samples are reserved to form an independent test set; the generalization performance of the obtained grain condition ensemble classifier is verified on the independent test set. The generalization performance of the grain condition status integrated classifier satisfies the following constraints: Based on the McDiarmid inequality, for a preset error fluctuation threshold The empirical error calculated by the grain condition status integrated classifier on the independent test set. Its mathematical expectation The deviation between them exceeds probability satisfy: ； in, Indicates the number of samples in the independent test set. Represents the natural exponential function; Based on Rademacher complexity, the true error of the grain condition integrated classifier satisfy: ； in, This represents the dimension of the feature vector input to the grain condition ensemble classifier. For asymptotic upper bounds, This indicates that it includes all growth rates not exceeding [a certain percentage]. A set of functions that are constant multiples of each other.
8. The grain depot condition prediction method based on error-driven ensemble learning according to claim 1, characterized in that, In step 5, the grain condition status integrated classifier outputs the probability that the real-time grain condition monitoring data belongs to the normal state, the overheating state, or the condensation state. When the probability of the overheating state or the condensation state exceeds the warning threshold, an abnormal grain condition warning is triggered.
9. The grain depot grain condition prediction method based on error-driven ensemble learning according to claim 1, characterized in that, The base model for the weak classifier can be any one of decision tree, random forest, XGBoost, or LightGBM.
10. A grain depot grain condition prediction system based on error-driven ensemble learning, characterized in that, The system is used to implement the grain depot grain condition prediction method based on error-driven ensemble learning as described in any one of claims 1 to 9, the system comprising: The data acquisition module is used to collect grain condition monitoring data from multiple monitoring points in the grain depot in a time sequence and determine the grain condition status at each monitoring moment. The grain condition monitoring data includes temperature and humidity data, and the grain condition status includes normal status, heat generation status, and condensation status. The dataset construction module is used to construct the original training dataset based on the grain condition monitoring data and its corresponding grain condition status at each monitoring time. The number of samples in the normal state is greater than the number of samples in the fever state and the number of samples in the condensation state. The dataset partitioning module is used to divide the original training dataset into multiple training subsets using stratified sampling techniques. The iterative optimization module is used to perform the following iterative optimization process: For each training subset, a weak classifier is trained to predict the grain condition status. Each weak classifier is applied to the remaining training subsets other than its own corresponding training subset to predict the grain condition status, identify the misjudged samples that were incorrectly predicted by the weak classifier, and construct the hard sample set corresponding to each weak classifier based on the misjudged samples. For each weak classifier, the set of hard samples is merged into the corresponding training subset to form an enhanced training subset. Based on the enhanced training subset, the corresponding weak classifier is updated. Repeat the above steps of predicting grain condition and updating the weak classifier, and iterate for multiple rounds until the preset iteration termination condition is met. The grain condition prediction module is used to integrate all the iteratively optimized weak classifiers to form the final grain condition integrated classifier, and to predict the real-time grain condition based on the real-time grain condition monitoring data and the grain condition integrated classifier.