Gradient decision tree-integrated Gaussian Bayesian warehouse cargo volume prediction method and system

Abstract

The invention discloses a Gaussian Bayesian warehouse cargo volume prediction method and system integrated with a gradient decision tree, and the method comprises the steps: carrying out the data preprocessing and feature selection dimension reduction of warehouse logistics data stored in an enterprise data warehouse, and obtaining a preprocessed feature data set; a K-Means SMOTE method is adopted to carry out data set balance processing on the preprocessed feature data set; adopting a cross validation grid search algorithm to obtain an optimal parameter combination of the base learner based on the gradient decision tree; based on the optimal parameter combination, using a base learner to perform feature importance analysis and feature selection dimension reduction on the balanced feature data set, and preliminarily predicting a warehouse goods quantity value; and taking the preliminary prediction result of the base learners as input data to train a Gaussian Bayesian model so as to probabilistically combine the outputs of the plurality of base learners and output a final prediction result. According to the technical scheme, the synthetic sample can be generated to balance the data set, so that the problem of class imbalance is reduced, and the prediction performance is improved.

Description

Technical Field

[0001] The present invention relates to the field of data analysis and data mining, and in particular to a Gaussian Bayesian warehouse cargo volume prediction method and system that integrates a gradient decision tree. Background Art

[0002] In modern supply chain management, accurate forecasting of warehouse cargo volume is a key link in improving enterprise operational efficiency, reducing inventory backlogs, and optimizing resource allocation. With the development of information technology, enterprises have begun to collect and store a large amount of warehouse logistics data, including multi-dimensional data such as inventory changes, sales records, historical demand, and supply chain delay information. These data make it possible to build efficient forecasting models, but also bring challenges in data processing and model building.

[0003] On the one hand, warehouse logistics data is known for its high-dimensional, multi-source, and heterogeneous characteristics. This complex data structure poses a severe challenge to the robustness of the prediction model. Specifically, high-dimensional data means that the feature space is huge, and there is no lack of redundant and irrelevant features. If these features are not effectively screened and integrated, they will not only increase the computational burden, but may also introduce noise, resulting in instability and deviation in the prediction results. Therefore, how to use advanced data preprocessing techniques, such as feature selection, dimensionality reduction, data cleaning, etc., to optimize the feature data set so that it is both concise and rich in effective information has become the primary task to improve the accuracy of predictions.

[0004] On the other hand, as a leader in the field of machine learning, the gradient decision tree (GBDT) has shown great potential in warehouse cargo volume forecasting with its powerful predictive ability and moderate interpretability. However, when dealing with data sets containing noise or imbalanced distribution, the GBDT model is prone to fall into the trap of overfitting, that is, the model over-learns the random fluctuations or deviations in the training data, resulting in poor performance on unseen data. In addition, although the GBDT algorithm is flexible and changeable, different categories of GBDT learners (such as XGBoost, LightGBM, etc.) have their own advantages in training efficiency, memory usage, feature processing capabilities, etc. How to intelligently select and integrate the advantages of these learners according to specific task requirements to build a more efficient and robust prediction model is an urgent problem to be solved in current research. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a Gaussian Bayesian warehouse cargo volume prediction method and system integrating a gradient decision tree, which deeply integrates the advantages of Gaussian Processes (GP) and gradient decision trees (GBDT) to achieve intelligent processing and efficient prediction of high-dimensional, multi-source, heterogeneous warehouse logistics data. The method includes the following steps:

[0006] Step S1: Perform data preprocessing and feature selection dimensionality reduction on the warehousing logistics data stored in the enterprise data warehouse to obtain a preprocessed feature data set;

[0007] Step S2: Use K-Means SMOTE method to balance the preprocessed feature data set;

[0008] Step S3: A cross-validation grid search algorithm is used to obtain the optimal parameter combination of the gradient decision tree-based base learners XGBoost, LightGBM, and CatBoost models;

[0009] Step S4: Based on the optimal parameter combination obtained in step S3, XGBoost, LightGBM, and CatBoost algorithms are used as base learners to perform feature importance analysis and feature selection dimensionality reduction on the balanced feature data set, and a preliminary prediction of the warehouse cargo volume value is made to obtain a preliminary prediction result;

[0010] Step S5: Establish a Gaussian Bayesian model, and use the preliminary prediction results of the base learners as input data to train the Gaussian Bayesian model, so as to probabilistically merge the outputs of multiple base learners and output the final prediction result information.

[0011] Beneficial effects:

[0012] 1. The present invention provides a Gaussian Bayesian warehouse cargo volume prediction method integrating gradient decision trees, which adopts K-Means SMOTE oversampling method to balance the data set. This method combines K-means clustering algorithm and synthetic minority oversampling technology (SMOTE) algorithm, aiming to balance the data set by generating synthetic samples, so as to reduce the class imbalance problem.

[0013] 2. The present invention adopts a method combining cross-validation (CV) and grid search algorithms to optimize the parameters of gradient decision tree-based base learners XGBoost, LightGBM, and CatBoost. The combination of cross-validation and grid search can automatically complete the tuning process of model parameters, reducing the time and cost of manual trial and error, making model tuning more efficient and systematic, while improving the interpretability and transparency of the model.

[0014] 3. In the process of using the base learner based on the gradient decision tree to preliminarily predict the cargo volume information, the present invention considers the contribution of each feature to the model prediction and assigns a corresponding weight value, and further performs feature selection according to the feature weight value, so as to optimize the model processing process and improve the accuracy and robustness of the model.

[0015] 4. The present invention adopts the Gaussian Bayesian algorithm (GNB) to combine the nonlinear decision-making ability of the gradient decision tree with the probabilistic inference ability of GNB, so that the model can make more comprehensive and accurate predictions after comprehensively considering all relevant factors; and Gaussian Bayes provides a method to measure the confidence of the model to help quantify uncertainty, thereby reducing the bias of the model and further improving the accuracy and robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The present invention is a flowchart of a Gaussian Bayesian warehouse cargo volume prediction method integrating a gradient decision tree.

[0017] Figure 2 The figure is a schematic diagram of the data preprocessing process according to an embodiment of the present invention.

[0018] Figure 3 Pseudocode flow chart for the K-Means SMOTE oversampling algorithm.

[0019] Figure 4 Flowchart of the operation of the cross-validation grid search algorithm.

[0020] Figure 5 Pseudocode flowchart for the Gaussian Bayes algorithm.

[0021] Figure 6 The present invention is a structural block diagram of a Gaussian Bayesian warehouse cargo volume prediction system integrating a gradient decision tree.

[0022] Description of reference numerals: 61, preprocessing module; 62, data balancing module; 63, parameter acquisition module; 64, data processing and prediction module; 65, optimization module DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0024] Embodiment 1

[0025] like Figure 1 As shown, a Gaussian Bayesian warehouse cargo volume prediction method with an integrated gradient decision tree provided in an embodiment of the present invention includes the following steps:

[0026] Step S1: Perform data preprocessing and feature selection dimensionality reduction on the warehousing logistics data stored in the enterprise data warehouse to obtain a preprocessed feature data set;

[0027] Step S2: Use K-Means SMOTE method to balance the preprocessed feature data set;

[0028] Step S3: A cross-validation grid search algorithm is used to obtain the optimal parameter combination of the gradient decision tree-based base learners XGBoost, LightGBM, and CatBoost models;

[0029] Step S4: Based on the optimal parameter combination obtained in step S3, XGBoost, LightGBM, and CatBoost algorithms are used as base learners to perform feature importance analysis and feature selection dimensionality reduction on the balanced feature data set, and a preliminary prediction of the warehouse cargo volume value is made to obtain a preliminary prediction result;

[0030] Step S5: Establish a Gaussian Bayesian model, and use the preliminary prediction results of the base learners as input data to train the Gaussian Bayesian model, so as to probabilistically merge the outputs of multiple base learners and output the final prediction result information.

[0031] In one embodiment, the above step S1: performing data preprocessing and feature selection dimensionality reduction on the warehousing logistics data stored in the enterprise data warehouse to obtain a processed feature data set specifically includes:

[0032] Step S11: performing data anomaly detection on the warehouse logistics data, including: using a missing value matrix or a heat map to visualize the abnormal data and analyze the distribution pattern of the abnormal data, and using a box plot to visualize the data distribution to determine outliers;

[0033] Step S12: Processing the detected abnormal data, including: processing missing data: deleting, interpolating or ignoring the missing data according to the number, proportion and distribution pattern of the missing values; processing outlier abnormal data: interpolating and replacing the outlier values;

[0034] Step S13: standardize and discretize the format and unit of the warehouse logistics data after abnormal data processing, convert the data containing units in the warehouse logistics data into a unified scale unit, and discretize the feature information with classification characteristics and continuous characteristics into the same interval category;

[0035] Step S14: Perform feature combination and selective dimensionality reduction on the features of the warehouse logistics data that have been standardized and discretized in terms of format and unit to obtain robust feature data, perform feature combination on the features that can be merged in the feature data to generate new interactive features, and then perform principal component analysis (PCA) on the features to select the features that are more important to the analysis target to form the final feature data set.

[0036] like Figure 2 The figure shows a schematic diagram of the data preprocessing process.

[0037] In one embodiment, the above step S2: using the K-Means SMOTE method to balance the preprocessed data set specifically includes:

[0038] Step S21: Clustering: Use K-means spatial clustering method to cluster the preprocessed data set into k groups of data, each group of data is called a cluster;

[0039] Step S22: Filtering: Filter the k clusters to retain the clusters with a higher proportion of minority class samples;

[0040] Step S23: Oversampling: Based on the characteristics and distribution of the existing minority class samples, SMOTE is applied to each cluster retained in step S22 for oversampling, and new synthetic sample data is generated and distributed to the clusters with sparse distribution of minority class samples to achieve a similar ratio of data quantity in each cluster and ensure data diversity. The generated new synthetic sample data has the same column structure as the preprocessed feature data and is consistent with the original data in distribution in feature space and category.

[0041] like Figure 3 Shown is a pseudocode flow chart of the K-Means SMOTE oversampling algorithm.

[0042] In one embodiment, the above step S3: using a cross-validation grid search algorithm to obtain the optimal parameter combination of the base learners XGBoost, LightGBM, and CatBoost models based on the gradient decision tree specifically includes:

[0043] Step S31: setting an initial selection list set of model parameters of base learners XGBoost, LightGBM, and CatBoost, and dividing the selection list into multiple parameter grid subsets using a Cartesian product operation;

[0044] Step S32: Divide the feature data set preprocessed in step S1 into an even number of n sub-data sets, use n / 2 of the sub-data sets to traverse all parameter grid subsets for cross-validation, select the remaining n / 2 sub-data sets as validation sets each time, and record the average performance metric of each initial selection list set on all validation sets;

[0045] Step S33: Select the parameter combination with the best score in all parameter grid subsets as the model parameters.

[0046] like Figure 4 Shown is a flowchart of the cross-validation grid search algorithm.

[0047] In one embodiment, the above step S4: based on the optimal parameter combination obtained in step S3, using XGBoost, LightGBM, and CatBoost algorithms as base learners, performs feature importance analysis and feature selection dimensionality reduction on the balanced feature data set, and preliminarily predicts the warehouse cargo volume value to obtain a preliminary prediction result, which specifically includes:

[0048] Step S41: Use the base learner to train and evaluate the feature importance of the feature data in the preprocessed feature data set, and output the weight value set of the base learner for each feature data. ;in , , Divided into XGBoost, LightGBM, and CatBoos base learners for each feature data weight value;

[0049] Step S42: Select feature data that contributes more to the base learner and remove the weight value in the base learner , , The feature data that appears in the smallest ten feature data in the dataset is combined with the high-weight feature data to construct a new feature data set. The base model based on the XGBoost, LightGBM, and CatBoost algorithms is established based on the new feature data. This step can reduce the complexity of the model and prevent the model from overfitting.

[0050] Step S43: The newly constructed feature data set is used as the input data of the established base model to perform preliminary data processing and result prediction to obtain the predicted warehouse cargo volume result data.

[0051] In one embodiment, the above step S5: establishing a Gaussian Bayesian model, using the preliminary prediction results of the base learner as input data to train the Gaussian Bayesian model, thereby probabilistically merging the outputs of multiple base learners and outputting the final prediction result information, specifically includes:

[0052] Step S51: Parameter learning: Based on the Bayesian theorem, the mean and variance of each storage volume feature under each date category are learned from the storage volume result data predicted in step S4. The mathematical expression of the Bayesian theorem is shown in formula (1) as follows:

[0053] (1)

[0054] in, is the posterior probability, which indicates the probability of category Y given the known feature X; is the likelihood probability, which indicates the probability of feature X under the condition of known category Y; It is the prior probability, which indicates the initial probability of category Y without considering any features; is the marginal probability of the data, which serves as a normalizing constant;

[0055] Step S52: Probability prediction: Based on the Gaussian distribution assumption, use the learned parameters to calculate the posterior probability of each date category, and select the category with the highest probability as the prediction output. The Gaussian distribution assumption represents formula (2) as follows:

[0056] (2)

[0057] in, represents the i-th feature; is the mean of the Gaussian distribution, indicating that the feature The average value of is the variance, indicating the value of this feature in The surrounding distribution range.

[0058] like Figure 5 Shown is a pseudocode flow chart of the Gaussian Bayes algorithm.

[0059] Embodiment 2

[0060] like Figure 6 As shown, an embodiment of the present invention provides a Gaussian Bayesian warehouse cargo volume prediction system integrating a gradient decision tree, including the following modules:

[0061] The preprocessing module 61 performs data preprocessing and feature selection and dimension reduction on the warehousing logistics data stored in the enterprise data warehouse to obtain a processed feature data set;

[0062] A data balancing module 62 is used to balance the processed feature data set using a K-means SMOTE oversampling algorithm;

[0063] A parameter acquisition module 63 is used to obtain the optimal parameter combination of the base learners XGBoost, LightGBM, and CatBoost models based on the gradient decision tree by using a cross-validation grid search algorithm;

[0064] The data processing and prediction module 64 uses XGBoost, LightGBM, and CatBoost algorithms as base learners based on the obtained optimal parameter combination to perform feature importance analysis and feature selection dimensionality reduction on the balanced feature data set, and preliminarily predicts the storage volume value to obtain a preliminary prediction result;

[0065] The optimization module 65 establishes a Gaussian Bayesian model, uses the preliminary prediction results as input data to train the Gaussian Bayesian model, thereby probabilistically combining the outputs of multiple base learners and outputting the final prediction result information. This can balance the characteristics of different base learners to improve the accuracy of the final prediction.

Claims

1. A Gaussian Bayesian warehouse cargo volume prediction method based on an integrated gradient decision tree, characterized in that: The specific steps include: Step S1: Perform data preprocessing and feature selection dimensionality reduction on the warehousing logistics data stored in the enterprise data warehouse to obtain a preprocessed feature data set; Step S2: Use K-Means SMOTE method to balance the preprocessed feature data set; Step S3: A cross-validation grid search algorithm is used to obtain the optimal parameter combination of the gradient decision tree-based base learners XGBoost, LightGBM, and CatBoost models; Step S4: Based on the optimal parameter combination obtained in step S3, XGBoost, LightGBM, and CatBoost algorithms are used as base learners to perform feature importance analysis and feature selection dimensionality reduction on the balanced feature data set, and a preliminary prediction of the warehouse cargo volume value is made to obtain a preliminary prediction result; Step S5: Establish a Gaussian Bayesian model, and use the preliminary prediction results of the base learners as input data to train the Gaussian Bayesian model, so as to probabilistically merge the outputs of multiple base learners and output the final prediction result information.

2. The Gaussian Bayesian warehouse cargo volume prediction method based on the integrated gradient decision tree according to claim 1 is characterized in that: The step S1 comprises: Step S11: performing data anomaly detection on the warehouse logistics data, including: using a missing value matrix or a heat map to visualize the abnormal data and analyze the distribution pattern of the abnormal data, and using a box plot to visualize the data distribution to determine outliers; Step S12: Processing the detected abnormal data, including: processing missing data: deleting, interpolating or ignoring the missing data according to the number, proportion and distribution pattern of the missing values; processing outlier abnormal data: interpolating and replacing the outlier values; Step S13: standardize and discretize the format and unit of the warehouse logistics data after abnormal data processing, convert the data containing units in the warehouse logistics data into a unified scale unit, and discretize the feature information with classification characteristics and continuous characteristics into the same interval category; Step S14: Perform feature combination and selective dimensionality reduction on the features of the warehouse logistics data that have been standardized and discretized in terms of format and unit to obtain robust feature data, perform feature combination on the features that can be merged in the feature data to generate new interactive features, and then perform principal component analysis on the features to select the features that are more important to the analysis target to form the final feature data set.

3. The Gaussian Bayesian warehouse cargo volume prediction method based on the integrated gradient decision tree according to claim 1 is characterized in that: The step S2 comprises: Step S21: Clustering: Use K-means spatial clustering method to cluster the preprocessed data set into k groups of data, each group of data is called a cluster; Step S22: Filtering: Filter the k clusters to retain the clusters with a higher proportion of minority class samples; Step S23: Oversampling: Based on the characteristics and distribution of the existing minority class samples, SMOTE is applied to each cluster retained in step S22 for oversampling, and new synthetic sample data is generated and assigned to the clusters where the minority class samples are sparsely distributed. The generated new synthetic sample data has the same column structure as the preprocessed feature data, and is consistent with the preprocessed feature data in feature space and category distribution.

4. The Gaussian Bayesian warehouse cargo volume prediction method based on the integrated gradient decision tree according to claim 1 is characterized in that: The step S3 comprises: Step S31: setting an initial selection list set of model parameters of base learners XGBoost, LightGBM, and CatBoost, and dividing the selection list into multiple parameter grid subsets using a Cartesian product operation; Step S32: Divide the data set preprocessed in step S1 into an even number of n sub-data sets, use n / 2 of the sub-data sets to traverse all parameter grid subsets for cross-validation, select the remaining n / 2 sub-data sets as validation sets each time, and record the average performance metric of each initial selection list set on all validation sets; Step S33: Select the parameter combination with the best score in all parameter grid subsets as the model parameters.

5. The Gaussian Bayesian warehouse cargo volume prediction method based on the integrated gradient decision tree according to claim 1 is characterized in that: The step S4 comprises: Step S41: Use the base learner to train and evaluate the feature importance of the feature data in the preprocessed feature data set, and output the weight value set of the base learner for each feature data. ;in , , Divided into XGBoost, LightGBM, and CatBoos base learners for each feature data weight value; Step S42: Select feature data that contributes more to the base learner and remove the weight value in the base learner , , The feature data that appears in the smallest ten feature data in the dataset are combined with other feature data to construct a new feature data set, and the base model based on the XGBoost, LightGBM, and CatBoost algorithms is established based on the new feature data; Step S43: The newly constructed feature data set is used as the input data of the established base model to perform preliminary data processing and result prediction to obtain the predicted warehouse cargo volume result data.

6. The Gaussian Bayesian warehouse cargo volume prediction method based on integrated gradient decision tree according to claim 1 is characterized in that: The step S5 comprises: Step S51: Parameter learning: Based on the Bayesian theorem, the mean and variance of each storage volume feature under each date category are learned from the storage volume result data predicted in step S4. The mathematical expression of the Bayesian theorem is shown in formula (1) as follows: (1) in, is the posterior probability, which indicates the probability of category Y given the known feature X; is the likelihood probability, which indicates the probability of feature X under the condition of known category Y; It is the prior probability, which indicates the initial probability of category Y without considering any features; is the marginal probability of the data, which serves as a normalizing constant; Step S52: Probability prediction: Based on the Gaussian distribution assumption, use the learned parameters to calculate the posterior probability of each date category, and select the category with the highest probability as the prediction output. The Gaussian distribution assumption represents formula (2) as follows: (2) in, represents the i-th feature; is the mean of the Gaussian distribution, indicating that the feature The average value of is the variance, indicating the value of this feature in The surrounding distribution range.

7. A Gaussian Bayesian warehouse cargo volume prediction system integrating gradient decision trees, characterized in that: Includes the following modules: The preprocessing module performs data preprocessing and feature selection and dimensionality reduction on the warehousing logistics data stored in the enterprise data warehouse to obtain a preprocessed feature data set; The data balancing module is used to balance the preprocessed feature data set using the K-means SMOTE oversampling algorithm; The parameter acquisition module is used to obtain the optimal parameter combination of the gradient decision tree-based base learners XGBoost, LightGBM, and CatBoost models using a cross-validation grid search algorithm; The data processing and prediction module uses XGBoost, LightGBM, and CatBoost algorithms as base learners based on the obtained optimal parameter combination to perform feature importance analysis and feature selection dimensionality reduction on the balanced feature data set, and preliminarily predicts the warehouse cargo volume value to obtain preliminary prediction results; The optimization module establishes a Gaussian Bayesian model and uses the preliminary prediction results as input data to train the Gaussian Bayesian model, so as to probabilistically merge the outputs of multiple base learners and output the final prediction result information.

Images