A Bayesian Optimization-Based RF Order Demand Forecasting Method

Abstract

This invention relates to a Bayesian optimization-based RF order demand forecasting method for simultaneously predicting the future demand or sales volume of multiple commodities. The method includes the following steps: 1) Preprocessing the historical order sales dataset and dividing it into training and test datasets as input and output of a random forest-based order demand forecasting model; 2) Obtaining the optimal hyperparameters of the order demand forecasting model using a Bayesian optimization method, thus obtaining the optimal order demand forecasting model; 3) Predicting the future sales volume of the ordered commodities based on the optimized order demand forecasting model. Compared with existing technologies, this invention has advantages such as high prediction accuracy, rapid optimization, and the ability to forecast multiple time series data.

Description

Technical Field

[0001] This invention relates to the field of market demand forecasting, and in particular to a Bayesian optimization-based RF order demand forecasting method. Background Technology

[0002] With the development of the internet and the accelerating pace of economic globalization, market competition has become increasingly fierce, and future market and customer demands will become more personalized and diversified. Existing production and operation models cannot predict the rapidly changing market or respond proactively and quickly to market demands. How to utilize scientific and reliable methods to accurately predict order demand, enabling companies or enterprises to escape the predicament of passively responding to the market and adapt to future market changes, has become a current challenge.

[0003] According to demand-driven theory, demand forecasting is the cornerstone of every link in the supply chain, including raw material procurement, production planning, inventory planning, and logistics planning. Inaccurate demand forecasting leads to discrepancies between production plans and actual demand. These discrepancies are amplified during transmission, increasing costs across the entire supply chain and causing significant losses for businesses. Therefore, accurate demand forecasting is crucial. Demand forecasting enables rapid responses to market fluctuations, meeting diverse user needs and allowing businesses to plan production, procurement, sales, and personnel and equipment allocation in advance. Based on demand forecasts, businesses can fully utilize their resources, reduce production costs, improve operational efficiency, and enhance their market competitiveness.

[0004] Order demand forecasting is modeled and predicted using time series analysis. The premise of using time series for forecasting is that a historical development can continue into the future, and data changes are gradual rather than abrupt. Time series forecasting can also be divided into single-step forecasting and multi-step forecasting based on the forecasting step size. Currently, existing time series analysis methods and machine learning-based time series analysis methods are widely used in order demand forecasting. However, existing time series analysis methods have certain limitations. Using this method requires certain pre-existing assumptions; secondly, existing methods have limited ability to fit nonlinear data, and the prediction accuracy often falls short of expectations. Machine learning models have high accuracy and good fitting and generalization capabilities for nonlinear data. Therefore, with the development and maturation of machine learning theory, using machine learning methods for time series data forecasting has become mainstream. The basic idea of ​​time series forecasting is to construct features by analyzing historical data as input to the model, build a loss function, and allow the model to iteratively train and learn to approximate the target. In practical applications, machine learning models are widely recognized. The selection and construction of features in machine learning models determine their accuracy. However, the input features of machine learning models are often manually constructed, and over-reliance on manually constructed features is one of the drawbacks of machine models.

[0005] With the advent of the big data era, the increasing variety of goods, and the continuous development of computer artificial intelligence, applying deep learning-based time analysis methods to order demand forecasting has become a popular research topic for scholars. However, many current studies focus on predicting the sales volume of a single product in an order. As the number of goods in an order increases, the ability to use a single model to predict the sales volume of multiple goods simultaneously has become a challenge in practical applications. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a Bayesian optimization-based RF order demand forecasting method.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] A Bayesian optimization-based RF order demand forecasting method, used to simultaneously predict the future demand or sales volume of multiple commodities, includes the following steps:

[0009] 1) Preprocess the historical order sales dataset and divide it into training and test datasets as input and output of the random forest-based order demand prediction model;

[0010] 2) Obtain the optimal hyperparameters of the order demand forecasting model through the Bayesian optimization method, and obtain the optimal order demand forecasting model;

[0011] 3) Based on the optimized order demand forecasting model, predict the future sales volume of the ordered goods.

[0012] In step 1), data preprocessing specifically includes the following steps:

[0013] 11) Collect historical order sales data and clean the data to remove duplicate and missing data, and then divide the dataset into training dataset and test dataset.

[0014] 12) Normalize the dataset to speed up the gradient descent solution to the optimal value and improve the model accuracy;

[0015] 13) Construct features, select step size t as the length of the sliding window, predict the order demand for the next month using historical sales data from the previous t-1 months, and divide each row of data using the sliding window;

[0016] 14) Use the product's SKU value, i.e. the product number, as a unique identifier to distinguish different products, and perform one-hot encoding on the product SKU values ​​in the order as an additional feature input.

[0017] In step 12), the min-max standardization method is used to transform the original historical order sales data linearly so that the values ​​are within the range of [0,1].

[0018] Step 2) of the above-mentioned modeling of the order demand forecasting model based on Bayesian optimization specifically includes the following steps:

[0019] 201) The optimization objective is to minimize the mean standard deviation (MSE) or maximize the mean R² score.

[0020] 202) Create a Bayesian object, set the optimization range of the parameters, and use the Bayesian optimization method to optimize the order demand forecasting model;

[0021] 203) Obtain the location of the maximum negative mean square error or the maximum R2 score, and the corresponding hyperparameter values;

[0022] 204) Input the optimal hyperparameters obtained by Bayesian optimization into the model, use the model to predict the test data, and evaluate the model.

[0023] In step 2), the training method for the order demand forecasting model specifically includes the following steps:

[0024] 211) Construct feature values ​​based on the data, and divide the dataset into training samples and test samples;

[0025] 212) Select M decision trees as weak classifiers, and use the bootstrap sampling method to sample M times from the training samples to form the training set {D1, D2, ... D} of the M decision trees. M}, corresponding to the decision tree {T1(θ),T2(θ),...T M (θ)};

[0026] 213) For each decision tree, when splitting a node, select n-dimensional features without replacement, traverse the mean squared error of each feature and select the feature corresponding to the smallest mean squared error as the best splitting feature.

[0027] 214) When the decision tree reaches the termination condition, i.e., the maximum depth of the tree (max-depth) or the mean squared error value reaches the set threshold, the splitting stops and the training is completed.

[0028] In step 213), the node splitting specifically involves:

[0029] Each decision tree uses a CART regression tree for regression prediction, and the minimum mean square error criterion is used to determine whether a node should split.

[0030] The expression for minimizing the mean square error m is:

[0031]

[0032] Where s is the sample set of the current node, A is the feature subset extracted from the sample set s, D1 and D2 are the subsets obtained by partitioning according to different features in the feature set, and y i c1 and c2 are the true output values ​​for each sample, and c1 and c2 are the mean output values ​​of the sample subsets D1 and D2, respectively.

[0033] In step 213), the features in the feature subset A are traversed, and the feature corresponding to the minimum mean square error is obtained as the best feature for dividing the node. This process is repeated continuously and a CART tree model is finally obtained. The predicted value is the mean value of the tree leaf node in the CART tree. The final RF prediction value is the weighted average of all tree prediction results.

[0034] In the Bayesian optimization method of step 204), the learning model for Bayesian optimization is specifically as follows:

[0035]

[0036] Where x represents the hyperparameter, X is the hyperparameter search space, f(x) is the objective function, and x* is the optimal hyperparameter;

[0037] The Gaussian process in Bayesian optimization methods is specifically as follows:

[0038] Assume that the objective function f(x) follows a Gaussian distribution, and that any finite number of random variables in the Gaussian process satisfy a joint probability distribution. There is observation noise that satisfies a normal distribution. The marginal likelihood is obtained from the likelihood function, and the mean and variance of the set of unknown functions are predicted from the Gaussian process.

[0039] The specific acquisition function in the Bayesian optimization method is as follows:

[0040] A confidence interval strategy is used to select the data acquisition function.

[0041] Step 3) further includes constructing a Bayesian-optimized RF order demand forecasting system, which includes:

[0042] Front-end page: Implemented using the Bootstrap framework and jQuery, including four functional blocks: user management, order data management, model selection and model training, and historical modeling records;

[0043] Backend Interface: The backend implementation uses Python, including data reading and saving, data processing and analysis, mathematical model building, and business logic processing. Data reading and storage are used to operate on data in the database, including reading and operating on historical order data and user data, and reading and saving process files generated during model training. Data processing and analysis and mathematical model building mainly use Python extension libraries, calling relevant APIs as needed to implement data processing and predictive model building.

[0044] Database: During the system development phase, an SQLite database is used to store information related to users of the system, order data, order backup data, and process files containing new data and charts generated during data processing, model training and prediction. It establishes a connection between the backend and the database, uses Model in SQLAlchemy to create data tables, and manages the database.

[0045] Compared with the prior art, the present invention has the following advantages:

[0046] This invention can simultaneously predict the future demand for multiple products in an order. The optimized RF model improves the accuracy of the prediction, and Bayesian optimization improves the speed of the roaming. At the same time, it can use the order demand prediction system to predict the products in different orders. Even non-professionals can operate according to the prompts on the system interface to complete the order prediction work. This system makes the order prediction work more digital and intelligent. Attached Figure Description

[0047] Figure 1 A schematic diagram of the modeling process for a Bayesian-optimized RF order demand forecasting model.

[0048] Figure 2 Overall design diagram for the order demand forecasting modeling system.

[0049] Figure 3 A schematic diagram is constructed for the features.

[0050] Figure 4 This is a flowchart of the random forest model algorithm.

[0051] Figure 5 This is a schematic diagram of the Bayesian optimization process.

[0052] Figure 6 This is a schematic diagram of k-fold cross-validation.

[0053] Figure 7 To optimize the Bayesian search process.

[0054] Figure 8 A comparison chart showing the prediction performance of Bayesian optimization and grid search.

[0055] Figure 9 A schematic diagram of the interface of the order demand forecasting modeling system. Detailed Implementation

[0056] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0057] This invention provides a Bayesian-optimized RF order demand forecasting system, the system comprising:

[0058] Order dataset preprocessing module: preprocesses the collected data, removes duplicate and incorrect data, divides the dataset into training and testing data, constructs features for the dataset, performs one-hot encoding on the product SKUs, and uses the preprocessed data as the input and output of the model;

[0059] RF order demand prediction model based on Bayesian optimization: Bayesian optimization is used to optimize the decision tree model with a large number of RF parameters, thereby finding a set of the best hyperparameters. These hyperparameters are then fed into the model to predict test set data and evaluate the model.

[0060] Order demand forecasting module: Based on the Bayesian optimized RF model, this module performs demand analysis for order modeling, including user management, order data management, historical record tracking, and the training of the most basic algorithm model and order demand forecasting functions. Finally, it completes the overall system design and development, realizing the functions in the demand analysis.

[0061] Based on the above system, the present invention also provides a Bayesian optimization-based RF order demand forecasting method, which includes the following steps:

[0062] 1) Data preprocessing, specifically including the following steps:

[0063] 11) Clean and overlay the data:

[0064] Check if there are duplicate or missing values ​​in the dataset, and discard duplicate and missing data. Table 1 is a schematic diagram of the original dataset. If the monthly sales demand of the product is to be predicted, the sales of the same SKU in each month need to be summed to obtain the total sales for each month.

[0065] Table 1 Description of the original dataset

[0066] ID Year Month Amount SKU 1 1 1 0.0 1235 2 1 2 0.23 1236 3 1 1 0.25 1235

[0067] 12) Data set standardization:

[0068] The most common methods for dataset standardization include "min-max standardization" and "Z-score standardization". Min-max standardization transforms the original data linearly to make its values ​​range between [0,1]. The min-max transformation function is as follows:

[0069]

[0070] Z-score standardizes the original data using the mean μ and variance σ. This method is suitable when the maximum and minimum values ​​of features or attributes are unknown. The transformation function is as follows:

[0071]

[0072] Normalized data helps to speed up the gradient descent solution to the optimal value and improve the accuracy of the model. Therefore, this invention uses the min-max normalization method to normalize the value of Amount so that the sales value falls within the range of [0,1].

[0073] 13) Construction of features:

[0074] Choosing the step size m as the length of the sliding window, i.e., time_step = m, for example, to predict monthly sales, we use the historical sales of the previous m-1 months to predict the order demand for the next month. Each row of data is divided using a sliding window. Assuming each row has n sets of data and the time step is m, then each row can yield n-m+1 data points. Figure 3 It is the process of constructing training samples through a sliding window.

[0075] 2) Based on the preprocessed dataset, establish a Bayesian-optimized RF order demand forecasting model, specifically including the following steps:

[0076] 21) For example Figure 4 As shown, the training process of the random forest algorithm is as follows:

[0077] 211) Construct feature values ​​based on the data, and divide the dataset into training samples and test samples;

[0078] 212) Select M decision trees as the number of weak classifiers, and use the bootstrap sampling method to sample M times from the training samples to form the training sets of the M decision trees, namely {D1, D2, ... D...}. M}, corresponding to the decision tree {T1(θ),T2(θ),...T M (θ)}

[0079] 213) For each decision tree, when splitting a node, select n-dimensional features without replacement, and iterate through the mean squared errors of each feature to select the feature with the smallest mean squared error as the optimal splitting feature. Then:

[0080]

[0081] 214) When the decision tree reaches the termination condition, i.e., the maximum depth of the tree (max-depth) or the mean squared error (m) reaches the set threshold (m0), the splitting stops;

[0082] 215) Use the trained model for prediction and evaluate the model using mean absolute error (MAE), mean absolute error (MSE), R2 score, and explained variance score. The expressions for each evaluation parameter are as follows:

[0083]

[0084]

[0085]

[0086]

[0087] (3) Obtain the optimal hyperparameters of the RF order demand forecasting model through Bayesian optimization. Figure 5The process of finding the optimal hyperparameters for Bayesian optimization involves a core Gaussian process and an extraction function. The Gaussian process updates the probabilistic surrogate model using the first t-1 sample points, thereby calculating the mean and variance for each sample point in the hyperparameter sample set. The extraction function obtains the new evaluation point for the t-th iteration based on the mean and variance from the previous step. If the set target value or the number of iterations is met, the globally optimal hyperparameter values ​​are found; otherwise, the Gaussian process continues. The specific process is described below:

[0088] (a) Bayesian optimization learning model:

[0089]

[0090] Where x represents the hyperparameter, X is the hyperparameter search space, and f(x) is the objective function.

[0091] (b) Gaussian process:

[0092] Assuming that f(x) follows a Gaussian distribution, a Gaussian process is assumed to have the following:

[0093] f(x)~GP(m(x),k(x,x'))

[0094] m(x)=E(f(x))

[0095] k(x,x')=E[(f(x)-m(x))(f(x'-m(x'))] (9)

[0096] Where k(x,x') is the covariance function and m(x) is the mean function, any finite number of random variables in a Gaussian process satisfy the following joint probability distribution:

[0097] p(f|X,θ)=N(0,∑) (10)

[0098] Where X is the training set, θ is the hyperparameter, and ∑ is the covariance matrix formed by k(x,x'). Assuming there exists observation noise ε such that the observed value y = f + ε, and the noise satisfies p(ε) = N(0,σ), the likelihood function can be obtained as follows:

[0099] p(y|f)=N(f,σ 2 I) (11)

[0100] Thus, the marginal likelihood can be calculated:

[0101] p(y|X,θ)=N(f,∑+σ 2 I) (12)

[0102] According to the Gaussian process:

[0103]

[0104] Where f * For predicted values, K ** =k(X) * ,X * ),X * To predict the hyperparameters of the input, the mean and variance of the predicted set of unknown functions are:

[0105]

[0106] (c) Acquisition Function

[0107] The acquisition function selects a new evaluation point from the candidate set; here, the confidence interval strategy (GP-UCB) is used for selection.

[0108]

[0109] The sampling function considers the ratio of the mean to the variance of the evaluation points. A larger mean indicates sampling in the region most likely to contain the global optimum, resulting in higher exploration; a larger variance indicates sampling in unexplored regions, resulting in higher exploration, and higher β. t It is a constant that balances the mean and variance.

[0110] (4) Establishment of a Bayesian-optimized RF order demand forecasting model. For example... Figure 1 The diagram illustrates the process of using Bayesian optimization to find the optimal hyperparameters in a Fire RF model. The specific steps include:

[0111] 41) Establish optimization objective: Perform 10 cross-validations on a single model. The optimization objective is to minimize the average mean squared error or maximize the average R² score of the regression model across the 10 cross-validations. The K-fold cross-validation process is as follows: Figure 6 As shown, cross-validation can be performed using cross_val_score introduced in sklearn.model_selection.

[0112] 42) Create a Bayesian object and set the optimization range for the parameters. Performing Bayesian optimization on the model requires installing the third-party package bayes_opt and importing BayesianOptimization from it to create a Bayesian object;

[0113] 43) Use the training data and Bayesian methods to train the RF model to obtain the optimal hyperparameters of the model;

[0114] 44) Returns the location of the maximum negative mean squared error or the maximum R2 score, along with the hyperparameter values;

[0115] 45) The optimal hyperparameters obtained by Bayesian optimization are substituted into the RF model, and the model is used to predict the test data.

[0116] 5) Design and implementation of an order demand forecasting modeling system. For example... Figure 2 The diagram shown is the overall design of the order demand forecasting and modeling system. The system is implemented using the Flask framework. The front-end is implemented using the Bootstrap framework and the jQuery language. Figure 2 The document showcases the four main functional modules of the front-end page: user management, order data management, model selection and training, and historical modeling records. The back-end is developed using Python, and AJAX technology is used for interaction between the front-end and back-end. Figure 2 The system also showcases the main tasks implemented by the backend, including data reading and storage, data processing and analysis, mathematical model building, and handling business logic. Data reading and storage are used to manipulate data in the database, such as reading and manipulating historical order data, user data, and reading and saving process files generated during model training. Data processing and analysis, as well as mathematical model building, primarily utilize multiple Python extension libraries such as NumPy, Scripy, Matplotlib, Pandas, sklearn, PyTorch, and Keras. Relevant APIs can be called as needed to implement data processing and predictive model building. During system development, an SQLite database was used to store user information, order data, order backup data, and process files such as new data and charts generated during data processing, model training, and prediction. Flask-SQLAlchemy was installed to establish a connection between the backend and the database. SQLAlchemy's Model component was used to create data tables and manage the database. Finally, the system's development tool was PyCharm, a Python integrated development environment with advantages such as syntax highlighting, ease of code debugging, and project management.

[0117] Example:

[0118] This invention verifies, through modeling and simulation experiments of Bayesian-optimized RF order demand forecasting, that the Bayesian-optimized RF demand forecasting model described above can predict the sales volume of various commodities and improve the accuracy of order demand forecasting.

[0119] 1. Introduction to the dataset and data preprocessing

[0120] 1.1 Introduction to the dataset

[0121] The dataset used in this example comes from the sales volume of orders from a manufacturing company. Table 2 shows a schematic diagram of the original dataset format. The dataset includes the order ID, the year and month in which the order was placed, the sales volume of the product, and the SKU (stock keeping unit). This dataset records the sales volume of 119 products over 12 months. Due to the different product types, customer demand for the products also varies. The demand for some products is relatively stable, with monthly sales fluctuating within a certain range. The demand for some products is intermittent or random, and the demand for some products varies depending on the season. There are complex relationships between these hundreds of similar time series. It is impractical to use traditional ARIMA or grey model algorithms for modeling. In order to explore the patterns among these hundreds of time series, i.e., the complex relationships between related time series, we first consider using machine learning methods to build a predictive model to predict the sales volume of these hundreds of products in the next month or several months.

[0122] Table 2. Schematic diagram of the original dataset

[0123] ID Year Month Amount SKU 1 1 1 0.0 1235 2 1 2 0.23 1236 3 1 1 0.25 1235

[0124] 1.2 Data Preprocessing

[0125] 1.2.1 Data Collection and Dataset Division

[0126] Use `df.duplicated` in Python to check for duplicate values ​​in the data; in this dataset, duplicate data is 0. If duplicate values ​​exist, delete the extra data. Check for missing values ​​in the dataset; if a missing value is in the `Amount` column, fill it with 0; if a missing value is in another column, discard that row. Since we need to predict the monthly sales of the company's products, we first need to sum the sales of the same SKU for each month to obtain the total sales for each month. Table 3 shows the results after data summing. The summation yields the total sales for each of the 12 months, with each row representing the sales for each SKU over the 12 months. We need to divide the dataset into training and validation data. When making single-step predictions of monthly sales, we choose to use the order sales of the previous 11 months as the training set and the data from the last month as the test set.

[0127] Table 3. Schematic diagram after data cleaning and collection.

[0128] Amount_1 Amount_2 Amount_3 Amount_4 Amount_5 … Amount_12 SKU 0 0.25 0.28 0.29 0.36 0.60 … 0.50 1235 1 0.30 0.35 0.38 0.32 0.33 … 0.36 1236 2 0.05 0.07 0.08 0.06 0.02 … 0.09 1237

[0129] 1.2.2 Dataset Standardization

[0130] The most common methods for dataset standardization include "min-max standardization" and "Z-score standardization". In this example, min-max standardization is used to transform the original data linearly so that its value range is between [0,1].

[0131] 1.2.3 Construction of Features

[0132] Choose a step size of m as the length of the sliding window, i.e., time_step = m. Use historical sales data from the previous m-1 months to predict order demand for the next month. Divide each row of data using a sliding window. Assuming each row has n sets of data and a time step of m, then each row can yield n-m+1 data points. Figure 3 It is the process of constructing training samples through a sliding window.

[0133] SKU is category data, used as a unique category identifier to distinguish different products. The sales data of each SKU may have different patterns of change. One-hot encoding is selected to encode this category data. Assuming there are 3 types of SKUs, Table 4 is a schematic diagram of one-hot encoding for each type of SKU.

[0134] Table 4 shows the one-hot encoding of SKUs.

[0135] Serial Number SKU SKU_1 SKU_2 SKU_3 Amount_1 0 1 1 0 0 0.25 1 2 0 1 0 0.06 2 3 0 0 1 0.5

[0136] 2. Experimental Results and Analysis

[0137] This example uses RandomForestRegressor, introduced from the sklearn.ensemble package, to build a random forest regression model. Random forests involve many parameters; the following is a description of the parameters involved in random forests and those that need to be adjusted:

[0138] n_estimators: The number of subtrees in the forest. The larger the n_estimators, the more iterations the weak learner needs, the greater the computational cost, and the easier it is to overfit.

[0139] oob_score: The score of the remaining samples in the random forest that were not used in training, reflecting the model's generalization ability.

[0140] criterion: The evaluation criterion for the feature. The default is the Gini coefficient, but information gain can also be used.

[0141] max_depth: The maximum depth of the decision tree, limiting the depth of subtrees. The default value is None, and commonly used values ​​are 10-100.

[0142] min_samples_split: Limits the minimum number of samples required for the sub-decision tree to continue splitting.

[0143] min_samples_leaf: Minimum number of samples in a leaf node, 1 is the default value.

[0144] max_features: The maximum number of features considered when partitioning RF nodes. auto is the default value.

[0145] Before setting the parameters for the prediction model, 10-fold cross-validation was performed on the training set using a single decision tree model and a random forest model. This demonstrated that the random forest model, based on the bagging ensemble approach, achieved higher prediction accuracy and better stability than the single decision tree model.

[0146] After Bayesian optimization, the RF demand forecasting model can find the optimal set of hyperparameters. Figure 7 This describes the Bayesian optimization process of the Random Forest (RF) model, finding the optimal result on the 28th attempt. Simulation experiments compared the Bayesian optimization method with the grid search method. Table 2 shows the parameter ranges for Bayesian optimization and grid search in the Random Forest model, and the resulting hyperparameter settings. Table 5 presents the evaluation metrics for the Random Forest model based on grid search and Bayesian optimization. Figure 8 This table compares the predicted results of the optimized Random Forest (RF) model with the actual values. Table 6 shows that the optimized RF model has a better fit to the predicted data. Because RF has many parameters, grid search optimizes several parameters simultaneously, which places high demands on computer hardware and takes a long time. Bayesian optimization, on the other hand, is much faster. Simulation results show that Bayesian optimization takes 91 seconds, while grid search takes 500 seconds to find just one parameter (n_estimators). Comparing the evaluation metrics and parameter optimization times, Bayesian optimization is faster and has stronger search capabilities than grid search. Figure 8 The results also show that the predicted data is close to the actual data, and the RF model based on Bayesian optimization has a stronger fitting ability compared to the grid search method. Experiments demonstrate that using Bayesian optimization to find parameters improves the prediction speed and accuracy of the RF model.

[0147] Table 5. Hyperparameter settings for the RF model based on grid search and Bayesian optimization.

[0148]

[0149] Table 6 Evaluation metrics for RF models based on grid search and Bayesian optimization

[0150]

[0151] like Figure 9 As shown, you can train the model by selecting the dataset and parameters. After training, you can click "Save" to save the model, or continue to modify the parameters to adjust the model. You can click the "View" button to view the prediction results.

Claims

1. A Bayesian optimization-based RF order demand forecasting method, used to simultaneously forecast the future demand or sales volume of multiple commodities, characterized in that... Includes the following steps: 1) Preprocess the historical order sales dataset and divide it into training and test datasets as input and output of the random forest-based order demand prediction model; 2) Obtain the optimal hyperparameters of the order demand forecasting model through the Bayesian optimization method, and obtain the optimal order demand forecasting model; 3) Based on the optimized order demand forecasting model, predict the future sales volume of the ordered goods; In step 1), data preprocessing specifically includes the following steps: 11) Collect historical order sales data and clean the data to remove duplicate and missing data, and then divide the dataset into training dataset and test dataset. 12) Normalize the dataset to speed up the gradient descent solution to the optimal value and improve the model accuracy; 13) Construct features, select step size t as the length of the sliding window, predict the order demand for the next month using historical sales data from the previous t-1 months, and divide each row of data into segments using the sliding window; 14) Use the product's SKU value, i.e., the product number, as a unique identifier to distinguish different products, and perform one-hot encoding on the product SKU values ​​in the order as an additional feature input; Step 2) of the above-mentioned modeling of the order demand forecasting model based on Bayesian optimization specifically includes the following steps: 201) The optimization objective is to minimize the mean standard deviation (MSE) or maximize the mean R² score. 202) Create a Bayesian object, set the optimization range of the parameters, and use the Bayesian optimization method to optimize the order demand forecasting model; 203) Obtain the location of the maximum negative mean square error or the maximum R2 score, and the corresponding hyperparameter values; 204) Input the optimal hyperparameters obtained by Bayesian optimization into the model, use the model to predict the test data, and evaluate the model. In step 2), the training method for the order demand forecasting model specifically includes the following steps: 211) Construct feature values ​​based on the data, and divide the dataset into training samples and test samples; 212) Select M decision trees as weak classifiers, and use the bootstrap sampling method to sample M times from the training samples to form the training set {D1, D2, ... D} of the M decision trees. M }, corresponding to the decision tree {T1(θ),T2(θ),...T M (θ)}; 213) For each decision tree, when splitting a node, select n-dimensional features without replacement, traverse the mean squared error of each feature and select the feature corresponding to the smallest mean squared error as the best splitting feature. 214) When the decision tree reaches the termination condition, i.e., the maximum depth of the tree (max-depth) or the mean squared error reaches the set threshold, the splitting stops and training is complete. In step 213), the node splitting specifically involves: Each decision tree uses a CART regression tree for regression prediction, and the minimum mean square error criterion is used to determine whether a node should split. In step 12), the min-max normalization method is used to transform the original historical order sales data linearly so that the values ​​are within the range of [0,1]. The expression for minimizing the mean square error m is: Where s is the sample set of the current node, A is the feature subset extracted from the sample set s, D1 and D2 are the subsets obtained by partitioning according to different features in the feature set, and y i For each sample, c1 and c2 are the true output values, and c1 and c2 are the mean output values ​​of sample subsets D1 and D2, respectively. In step 213), the features in the feature subset A are traversed, and the feature corresponding to the minimum mean square error is obtained as the best feature for dividing the node. This process is repeated continuously and a CART tree model is finally obtained. The predicted value is the mean value of the tree leaf node in the CART tree. The final RF prediction value is the weighted average of all tree prediction results. In the Bayesian optimization method of step 204), the learning model for Bayesian optimization is specifically as follows: Where x represents the hyperparameter, X is the hyperparameter search space, f(x) is the objective function, and x* is the optimal hyperparameter; The Gaussian process in Bayesian optimization methods is specifically as follows: Assume that the objective function f(x) follows a Gaussian distribution, and that any finite number of random variables in the Gaussian process satisfy a joint probability distribution. There is observation noise that satisfies a normal distribution. The marginal likelihood is obtained from the likelihood function, and the mean and variance of the set of unknown functions are predicted from the Gaussian process. The specific acquisition function in the Bayesian optimization method is as follows: A confidence interval strategy is used to select the data acquisition function; Step 3) further includes constructing a Bayesian-optimized RF order demand forecasting system, which includes: Front-end page: Implemented using the Bootstrap framework and jQuery, including four functional blocks: user management, order data management, model selection and model training, and historical modeling records; Backend Interface: The backend implementation uses Python, including data reading and saving, data processing and analysis, mathematical model building, and business logic processing. Data reading and storage are used to operate on data in the database, including reading and operating on historical order data and user data, and reading and saving process files generated during model training. Data processing and analysis and mathematical model building mainly use Python's extension libraries, calling relevant APIs as needed to implement data processing and predictive model building. Database: During the system development phase, an SQLite database is used to store information related to users of the system, order data, order backup data, and process files containing new data and charts generated during data processing, model training and prediction. It establishes a connection between the backend and the database, uses Model in SQLAlchemy to create data tables, and manages the database.

Images