Material demand prediction method and system based on iWOA-iTransform model

Abstract

The invention discloses a material demand prediction method and system based on an iWOA-iTransform model, and the method comprises the steps: carrying out the collection and preprocessing of the time series data of material consumption in the production and manufacturing process of an equipment manufacturing enterprise, and obtaining a data set; an improved whale optimization algorithm iWOA and an iTransform model are introduced, and the improved whale optimization algorithm iWOA is used to carry out hyper-parameter optimization on the iTransform model; based on the obtained hyper-parameter optimal solution, constructing an improved iTransform model, and based on the improved iTransform model, capturing association between time sequences in the data set; and based on association between time sequences in the data set, through training an improved iTransform model, predicting the material demand, and obtaining a final material demand value.

Description

Technical Field

[0001] The present invention belongs to the field of equipment manufacturing, and specifically relates to a material demand forecasting method and system based on an iWOA-iTransformer model. Background Art

[0002] In actual production situations, material inventory shortages will affect the production schedule of the production line, leading to delayed delivery cycles, while excess material inventory will increase inventory management costs and product costs. In order to effectively reduce the impact of material shortages and over-purchasing, help enterprises reduce inventory costs, improve enterprise production efficiency and risk resistance, and achieve the goal of reducing costs and increasing efficiency, enterprises urgently need more accurate prediction models to guide material procurement. According to the literature (Xin Wei. Application of LSTM-Prophet Hybrid Model in Material Reserve Demand Forecasting [J]. Modern Computer, 2024, 30(04): 53-7.), it can be seen that traditional time series prediction uses models based on statistical methods, such as Auto Regressive Model (AR), Moving Average Model (MA), Autoregressive Integrated Moving Average Model (ARIMA), which are all unit prediction methods based on autoregression and are mainly used in the fitting of single variables and linear time series. However, actual problems are mostly complex problems with multiple variables and nonlinearity. Therefore, prediction methods based on machine learning have gradually replaced traditional time series prediction methods, such as recursive neural networks (RNN), long short-term memory neural networks (LSTM), Transformer, support vector machines (SVM), etc.

[0003] With the continuous development of prediction methods based on machine learning, the literature (Bai Chaoyang, Song Linjie, Li Xiaolin. Material demand combination prediction model based on EMD-PSO-LSSVR [J]. Statistics and Decision, 2018, 34 (18): 185-8.) proposed a material demand prediction model based on EMD-PSO-LSSVR. By using the empirical mode decomposition (EMD) method, the non-stationary time series is decomposed into a series of intrinsic mode functions (IMFs) and a residual term (Res). Then, combined with the actual business, each IMF is synthesized into high-frequency and low-frequency parts, representing short-term fluctuations and long-term trends, respectively, to mine more information. Combined prediction is then performed with the least squares support vector regression (LSSVR) model, which can efficiently predict the non-stationary material demand time series. The literature (LIUJ, LIN L, LI Z, et al. Spare aeroengine demand prediction model based on deepCroston method[J]. Journal of Aerospace Information Systems, 2020, 17(2): 125-33.) proposed a spare aeroengine demand prediction method based on the Deep Croston method. It adopted the Croston framework to process intermittent demand characteristics and used the deep learning network model for prediction, which achieved significant results in intermittent demand prediction. The literature (Xiong Zhiyao, Li Lin. Retail fresh product inventory demand forecasting based on SARIMA-LSTM [J]. Logistics Technology, 2022, 45(03): 21-4+8.) proposed a retail fresh product inventory demand forecasting model based on SARIMA-LSTM. It considers demand influencing factors such as on-time delivery rate, comprehensive cost, temperature conditions, sales amount, etc. from three aspects: suppliers, retailers, and users. It combines the Bayesian optimization algorithm to select hyperparameters for LSTM, corrects the error sequence between the SARIMA model prediction value and the actual value, and obtains the prediction value, which provides a reasonable reference for the inventory prediction of fresh goods in the retail industry.The literature (LIU R, VAKHARIA V. Optimizing Supply Chain Management Through BO-CNN-LSTM for Demand Forecasting and Inventory Management [J]. Journal of Organizational and EndUser Computing (JOEUC), 2024, 36 (1): 1-25.) proposed a BO-CNN-LSTM model, which uses Bayesian optimization for hyperparameter adjustment, CNN for spatiotemporal feature extraction, and LSTM for modeling sequential data, thereby controlling costs and improving operational efficiency in supply chain management.

[0004] Through the above analysis, it is found that in order to ensure the accuracy of the prediction results, the existing technology uses a relatively complex combination model. Although these models contribute to improving the prediction accuracy, the complexity of the model brings high computational costs and resource consumption. Considering the particularity of time series data, the iTransformer model based on the self-attention mechanism can capture the long-range dependencies and complex nonlinear relationships in the time series, and has the advantages of parallel processing and reducing gradient disappearance. Therefore, we propose a material demand forecasting method and system based on the iWOA-iTransformer model. Summary of the invention

[0005] In view of the phenomenon that the existing technology usually relies on complex models and algorithms when processing material demand forecasting, although it can improve the prediction accuracy, the computing efficiency is significantly reduced in large-scale data processing and real-time application scenarios, and it is difficult to achieve a balance between accuracy and efficiency. The present invention proposes a material demand forecasting method and system based on the iWOA-iTransformer model, which can not only accurately capture the complex changing trends of material demand, but also effectively improve the computing efficiency and the real-time prediction capability of the model, and realize the unity of accuracy and efficiency in material demand forecasting.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] The material demand forecasting method based on the iWOA-iTransformer model includes the following steps:

[0008] Collect and preprocess the time series data of material consumption in the production process of equipment manufacturing enterprises to obtain a data set;

[0009] Introduce the improved whale optimization algorithm iWOA and iTransformer model, and use the improved whale optimization algorithm iWOA to optimize the hyperparameters of the iTransformer model;

[0010] An improved iTransformer model is constructed based on the optimal solution of hyperparameters, and the correlation between time series in the data set is captured based on the improved iTransformer model;

[0011] Based on the correlation between time series in the data set, the material demand is predicted by training the improved iTransformer model to obtain the final material demand value.

[0012] Preferably, the improved whale optimization algorithm iWOA includes: a prey encirclement stage, a spiral predation stage, a random search stage, and a differential evolution algorithm introduction stage;

[0013] Among them, the process of surrounding the prey stage is expressed as:

[0014] D=|C·X * (t)-X(t)|

[0015] X(t+1)=X * (t)-A·D

[0016] Among them, D is the hunting step length of humpback whales and prey, t is the current iteration number, X * (t) is the position vector of the prey, X(t) is the position vector of the humpback whale, and A and C are coefficient vectors;

[0017] The process of the spiral predation stage is expressed as:

[0018] X(t+1)=D'·e bl ·cos(2πl)+X * (t)

[0019] D'=|X * (t)-X(t)|

[0020] Where D' is the distance between the current individual and the prey, b is the constant defining the logarithmic spiral equation, and l is a random number in the range of [-1,1];

[0021] The process of random search phase is expressed as:

[0022] D=|C·X rand (t)-X(t)|

[0023] X(t+1)=X rand (t)-A·D

[0024] Among them, X rand (t) is the position of a random individual in the group of whales;

[0025] The process of introducing the differential evolution algorithm stage is expressed as:

[0026] Set the mutation factor F and crossover probability CR to get the mutation strategy:

[0027] U i (j) = X * (j)+F·(X r1 (j)-X r2 (j))

[0028] Among them, i is the current iteration number, j is the current whale individual, and U i (j) is the new offspring generated by the j-th whale individual in the i-th iteration, r1 and r2 are two random numbers and satisfy r1≠r2≠i, X * (j) is the most abundant individual in the current population, are different individuals randomly selected from the population, and the parameter j is set rand represents a random number in the range of 1-n, where n is the number of whales in the population, when r≤CR||j=j rand Then a crossover mutation operation is performed to generate new individuals.

[0029] Preferably, the method for optimizing the hyperparameters of the iTransformer model using the improved whale optimization algorithm iWOA includes:

[0030] Through differential evolution, the original population is mutated and hybridized to generate new individuals and obtain the optimal solution for the hyperparameters.

[0031] Preferably, the improved iTransformer model includes: an embedding layer, a multi-head self-attention layer, a normalization layer, and a feedforward neural network layer;

[0032] Among them, the embedding layer converts the preprocessed data into vectors and implements the embedding operation based on the multi-layer perceptron MLP;

[0033] The multi-head self-attention layer is initialized with a randomly set weight matrix W. Q , W K and W V , the input time series data and the weight matrix are processed into query vector Q, keyword vector K and value vector V in the form of dot product, the query vector Q, keyword vector K and value vector V are projected h times and then the Attention calculation is performed respectively, and the Contact function is used to splice the attention calculation results in the multi-dimensional space, and finally the attention results of all subspaces are connected to obtain the final output result of the multi-head self-attention layer;

[0034] The normalization layer applies normalization to the series representation of individual variables, so that the characteristics of all variables are in a relatively uniform distribution;

[0035] The feedforward neural network layer is implemented using a multi-layer perceptron (MLP), which allows the feedforward network to act on the entire sequence and extract the intrinsic properties of the sequence.

[0036] Preferably, based on the improved iTransformer model, the method for capturing the correlation between time series in the data set includes:

[0037] The improved iTransformer model maps the entire time series of variables into the smallest modeling unit - words, and uses the variable as the main body to naturally mine multivariate associations based on words through the attention mechanism;

[0038] The differences caused by different units between variables are eliminated through the cooperation of the feedforward network and the normalization layer.

[0039] The present invention also provides a material demand forecasting system based on the iWOA-iTransformer model, including: an acquisition module, an optimization module, a capture module and a forecasting module;

[0040] The acquisition module is used to collect and preprocess the time series data of material consumption in the production process of equipment manufacturing enterprises to obtain a data set;

[0041] The optimization module is used to introduce the improved whale optimization algorithm iWOA and the iTransformer model, and use the improved whale optimization algorithm iWOA to optimize the hyperparameters of the iTransformer model;

[0042] The capture module is used to build an improved iTransformer model based on the optimal solution of hyperparameters, and to capture the correlation between time series in the data set based on the improved iTransformer model;

[0043] The prediction module is used to predict material demand based on the correlation between time series in the data set by training the improved iTransformer model to obtain the final material demand value.

[0044] Preferably, the improved whale optimization algorithm iWOA includes: a prey encirclement stage, a spiral predation stage, a random search stage, and a differential evolution algorithm introduction stage;

[0045] Among them, the process of surrounding the prey stage is expressed as:

[0046] D=|C·X * (t)-X(t)|

[0047] X(t+1)=X * (t)-A·D

[0048] Among them, D is the hunting step length of humpback whales and prey, t is the current iteration number, X* (t) is the position vector of the prey, X(t) is the position vector of the humpback whale, and A and C are coefficient vectors;

[0049] The process of the spiral predation stage is expressed as:

[0050] X(t+1)=D'·e bl ·cos(2πl)+X * (t)

[0051] D'=|X * (t)-X(t)|

[0052] Where D' is the distance between the current individual and the prey, b is the constant defining the logarithmic spiral equation, and l is a random number in the range of [-1,1];

[0053] The process of random search phase is expressed as:

[0054] D=|C·X rand (t)-X(t)|

[0055] X(t+1)=X rand (t)-A·D

[0056] Among them, X rand (t) is the position of a random individual in the group of whales;

[0057] The process of introducing the differential evolution algorithm stage is expressed as:

[0058] Set the mutation factor F and crossover probability CR to get the mutation strategy:

[0059] U i (j) = X * (j)+F·(X r1 (j)-X r2 (j))

[0060] Among them, i is the current iteration number, j is the current whale individual, and U i (j) is the new offspring generated by the j-th whale individual in the i-th iteration, r1 and r2 are two random numbers and satisfy r1≠r2≠i, X * (j) is the most abundant individual in the current population, are different individuals randomly selected from the population, and the parameter j is set rand represents a random number in the range of 1-n, where n is the number of whales in the population, when r≤CR||j=j rand Then a crossover mutation operation is performed to generate new individuals.

[0061] Preferably, the process of optimizing the hyperparameters of the iTransformer model using the improved whale optimization algorithm iWOA includes:

[0062] Through differential evolution, the original population is mutated and hybridized to generate new individuals and obtain the optimal solution for the hyperparameters.

[0063] Preferably, the improved iTransformer model includes: an embedding layer, a multi-head self-attention layer, a normalization layer, and a feedforward neural network layer;

[0064] Among them, the embedding layer converts the preprocessed data into vectors and implements the embedding operation based on the multi-layer perceptron MLP;

[0065] The multi-head self-attention layer is initialized with a randomly set weight matrix W. Q , W K and W V , the input time series data and the weight matrix are processed into query vector Q, keyword vector K and value vector V in the form of dot product, the query vector Q, keyword vector K and value vector V are projected h times and then the Attention calculation is performed respectively, and the Contact function is used to splice the attention calculation results in the multi-dimensional space, and finally the attention results of all subspaces are connected to obtain the final output result of the multi-head self-attention layer;

[0066] The normalization layer applies normalization to the series representation of individual variables, so that the characteristics of all variables are in a relatively uniform distribution;

[0067] The feedforward neural network layer is implemented using a multi-layer perceptron (MLP), which allows the feedforward network to act on the entire sequence and extract the intrinsic properties of the sequence.

[0068] Preferably, based on the improved iTransformer model, the method for capturing the correlation between time series in the data set includes:

[0069] The improved iTransformer model maps the entire time series of variables into the smallest modeling unit - words, and uses the variable as the main body to naturally mine multivariate associations based on words through the attention mechanism;

[0070] The differences caused by different units between variables are eliminated through the cooperation of the feedforward network and the normalization layer.

[0071] Compared with the prior art, the present invention has the following beneficial effects:

[0072] The present invention is a material demand forecasting method based on iWOA-iTransformer. The improved whale optimization algorithm (iWOA) with the idea of ​​differential evolution is introduced to increase the types of candidate solutions of the population and improve the optimization ability of the whale optimization algorithm. The iWOA algorithm is used to optimize the hyperparameters of the iTransformer model, so that the prediction effect of the iTransformer model is optimal. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0074] Figure 1 It is a schematic diagram of the flow of a material demand forecasting method based on the iWOA-iTransformer model according to an embodiment of the present invention;

[0075] Figure 2 A flowchart of an improved whale optimization algorithm according to an embodiment of the present invention;

[0076] Figure 3 is a structural diagram of an iTransformer model according to an embodiment of the present invention;

[0077] Figure 4 This is a comparison chart of the original data of materials 1-4 in the embodiment of the present invention, the iTransformer model predicted data, and the iWOA-iTransformer model predicted data;

[0078] Figure 5 This is a comparison chart of the original data of materials 5-8 in the embodiment of the present invention, the iTransformer model predicted data, and the iWOA-iTransformer model predicted data;

[0079] Figure 6 This is a comparison chart of the original data of materials No. 9-12, the iTransformer model prediction data, and the iWOA-iTransformer model prediction data of the embodiments of the present invention;

[0080] Figure 7 The experimental comparison results of the embodiments of the present invention are shown in Figure (a), where Figure (b) is a comparison of the MAE and RMSE evaluation indicators, and Figure (c) is a comparison of the MAPE and R 2 Comparison of evaluation indicators. DETAILED DESCRIPTION

[0081] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0082] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0083] Embodiment 1

[0084] like Figure 1 As shown, the material demand forecasting method based on the iWOA-iTransformer model includes the following steps:

[0085] Collect and preprocess the time series data of material consumption in the production process of equipment manufacturing enterprises to obtain a data set;

[0086] Introduce the improved whale optimization algorithm iWOA and iTransformer model, and use the improved whale optimization algorithm iWOA to optimize the hyperparameters of the iTransformer model;

[0087] An improved iTransformer model is constructed based on the optimal solution of hyperparameters, and the correlation between time series in the data set is captured based on the improved iTransformer model;

[0088] Based on the correlation between time series in the data set, the material demand is predicted by training the improved iTransformer model to obtain the final material demand value.

[0089] The specific implementation process is:

[0090] Step 1: Obtain material usage data using the Alibaba Cloud Infrastructure Supply Chain Inventory Management Decision Dataset. The original data set is stored according to the minimum inventory unit of the material, and the data set is sorted to obtain the total daily usage of each material.

[0091] Step 2: Preprocess the data in the dataset, such as deleting invalid, abnormal, or missing data or using a smooth interpolation method to preprocess the data. Finally, 12 materials are obtained as experimental objects, named from 1 to 12.

[0092] Step 3: Normalize the processed data set and divide it into training set, validation set and test set in a ratio of 8:2:1.

[0093] Step 4: Introduce the iTransformer model as the objective function of the swarm intelligence optimization algorithm to evaluate the fitness of each solution of the swarm intelligence algorithm.

[0094] like Figure 2 As shown, step five, introduce the improved whale optimization algorithm (iWOA), initialize the population, define the population size, the upper and lower bounds of the solution space, the objective function and the initial solution set, and determine the current optimal solution by calculating the fitness.

[0095] Step 5: Encircling the prey. Assuming that the current optimal solution is the target prey position, update the positions of other individuals in the whale group according to the position of the current optimal solution. The process can be expressed as:

[0096] D=|C·X * (t)-X(t) (0)

[0097] X(t+1)=X * (t)-A·D (2)

[0098] Where D is the hunting step length of the humpback whale and its prey, t is the current iteration number, X * (t) is the position vector of the prey, and X(t) is the position vector of the humpback whale. A and C are coefficient vectors, and their calculation method is:

[0099] A=2·a·ra (3)

[0100] C=2r (4)

[0101]

[0102] Where a is the linear convergence factor, r is a random vector in the range [0,1], T max is the maximum number of iterations. According to the linear decreasing property of a, a decreases linearly from 2 to 0 during the iteration process.

[0103] Step 6: Spiral predation stage. By calculating the distance between the current individual and the prey and constructing a spiral equation between the two, the spiral predation behavior of the whale is simulated. The process can be expressed as:

[0104] X(t+1)=D'·e bl ·cos(2πl)+X * (t) (6)

[0105] D'=|X * (t)-X(t) (7)

[0106] Where D' is the distance between the current individual and the prey, b is the constant that defines the logarithmic spiral equation, and l is a random number in the range of [-1,1]. Since whale individuals will perform both encircling prey and spiraling during the predation process, it is assumed that there is a 50% probability of choosing between these two actions during the optimization period. When the random probability p < 0.5, the killer whale will encircle the prey, and when the random probability p ≥ 0.5, the killer whale will perform spiraling, and the final mathematical expression is:

[0107]

[0108] Where p is the random probability of an event occurring, and its value range is [0,1].

[0109] Step 7: Random search phase. Enhance the global search capability by randomly searching for high-quality prey in the global range. This process can be expressed as:

[0110] D=|C·X rand (t)-X(t)| (9)

[0111] X(t+1)=X rand (t)-A·D (10)

[0112] Where X rand (t) is the position of a random individual in the whale group.

[0113] Step 8: Introduce the differential evolution algorithm to optimize the solution set. Set the mutation factor F and crossover probability CR to obtain the mutation strategy as follows:

[0114] U i (j) = X * (j)+F·(X r1 (j)-X r2 (j)) (11)

[0115] Where i is the current iteration number, j is the current whale individual, and U i (j) is the new offspring generated by the jth whale individual in the i-th iteration, r1 and r2 are two random numbers and satisfy r1≠r2≠i. * (j) is the most abundant individual in the current population, are different individuals randomly selected from the population. Set the control variable λ to control the solution search method. The mathematical expression is:

[0116]

[0117] Where t is the current iteration number, t max is the maximum number of iterations.

[0118] Determine the relationship between the random probability p and the control variable λ. Set the parameter jrand represents a random number in the range of 1-n, where n is the number of whales in the population. When p≤λ, if r≤CR||j=j rand Then perform crossover mutation operation to generate new individuals, otherwise enter the random search phase of step 7; when p>λ, if r≤0.5 is satisfied, enter the prey encirclement phase of step 5, otherwise enter the spiral predation phase of step 6. By calculating the individual fitness, update the optimal solution until the iteration ends.

[0119] Step 9: Use the optimal solution obtained in step 8 as the parameters of the iTransformer model, and build the iTransformer model for training and prediction.

[0120] Among them, the improved whale optimization algorithm (iWOA) is introduced to solve the problem that the whale optimization algorithm is prone to fall into the local optimal solution through differential evolution. Through iWOA, the parameters of the iTransformer model are optimized, so that the iTransformer model can maximize the long-term dependence of data in the time series and improve the prediction accuracy.

[0121] The improved whale optimization algorithm (iWOA) is introduced to solve the problem that the whale optimization algorithm is prone to fall into the local optimal solution through differential evolution. It uses the idea of ​​differential evolution to mutate and hybridize the original population to generate new individuals, thereby enriching the population diversity. The process includes:

[0122] Set the mutation factor F and crossover probability CR to get the mutation strategy:

[0123] U i (j) = X * (j)+F·(X r1 (j)-X r2 (j))

[0124] Where i is the current iteration number, j is the current whale individual, and U i (j) is the new offspring generated by the jth whale individual in the i-th iteration, r1 and r2 are two random numbers and satisfy r1≠r2≠i. * (j) is the best individual in the current population, are different individuals randomly selected from the population. Set the parameter j rand Represents a random number in the range 1-n, where n is the number of whales in the population.

[0125] When r≤CR||j=j rand Then a crossover mutation operation is performed to generate new individuals.

[0126] Using the iTransformer model to maximize the long-term dependencies of time series data specifically includes:

[0127] The entire time series of the variable is mapped to the smallest modeling unit-word through the iTransformer model, and the multivariate association in word units is naturally mined through the attention mechanism with the variable as the main body. The differences caused by different units between variables are eliminated through the cooperation of the feedforward network and the normalization layer. It solves the problem that the traditional method represents all variables as one word at the same time point, making the original variables mixed into multi-dimensional characteristics and difficult to capture the association between variables. Specifically, in the material requirement planning system (MRP) and enterprise resource planning system (ERP) of equipment manufacturing enterprises, the in-and-out situation of materials in a specified time range and specified category can be obtained, and the material usage on a specified date can be obtained by subtracting the material requisition value from the material return value. This method is used to obtain the usage of different models of gears and button station accessories of a certain enterprise from June 4, 2018 to March 1, 2021, and it is used as a data set through data preprocessing.

[0128] like Figure 3 As shown, the iTransformer model includes:

[0129] Embedding layer: Convert the preprocessed data into vectors and implement the embedding operation based on the multi-layer perceptron (MLP).

[0130] Multi-head self-attention layer: through the initial randomly set weight matrix W Q , W K and W V , the input time series data and the weight matrix are processed into query vector Q, keyword vector K and value vector V through dot product.

[0131]

[0132] After Q, K, and V are projected h times, Attention calculations are performed respectively, and the Contact function is used to splice the attention calculation results in the multi-dimensional space. Finally, the attention results of all subspaces are connected to obtain the final output result of the multi-head self-attention layer. The calculation process is as follows:

[0133]

[0134] head i =Attention(QW Q i ,KW K i ,VW V i) (15)

[0135] MultiHead(Q,K,V)=Contact(head1,...,head h )W O (16)

[0136] Among them: the softmax function is a normalized calculation function, d k is the dimension of the keyword vector K, W Q i , W K i and W V i are the transformation matrices of Q, K, and V in the i-th attention head, head i is the value of the i-th attention head, h is the number of multi-head attention, W O is the weight matrix corresponding to the output matrix.

[0137] Normalization layer: Normalization is applied to the series representation of individual variables, so that the characteristics of all variables are in a relatively uniform distribution, which improves the model's ability to solve non-stationary problems. The processing formula is as follows:

[0138]

[0139] Feedforward neural network layer: MLP is used to implement the feedforward network, which acts on the entire sequence and can extract the intrinsic properties of the sequence, thereby improving the generalization of the model on other sequences.

[0140] Set window size window = 32, iteration number epoch = 100, batch size batch_size = 32; use the parameters optimized in step 8: attention mechanism model dimension d_model, number of attention heads n_heads, fully connected layer dimension d_ff, number of encoder layers e_layers, dropout rate. Traverse the training data in each epoch and perform forward propagation for each batch. Calculate the loss with the obtained prediction results and actual labels. Back-propagate the loss and update the model parameters through the Adam optimizer. Record the training loss and validation loss of the model at the end of each epoch. Save the model with the smallest validation set loss as the final model during training.

[0141] Load the trained optimal model, set the prediction sequence length length_size = 7 to predict the usage of materials in the next week, and take the last known time window data from the most recent training data as the input of the model. Convert the input data and the corresponding time features into tensors that match the model and pass them into the model to perform forward propagation. Obtain the future prediction value through decoder operation and denormalization.

[0142] Step 10: After model training and prediction in step 9, the final prediction result is obtained, using the mean absolute error MAE, root mean square error RMSE, mean absolute percentage error MAPE and goodness of fit R 2 Four evaluation indicators are used to evaluate the prediction effect of the model. The calculation formula is as follows:

[0143]

[0144] Where n is the number of samples, y i is the actual value, is the sample mean, is the predicted value. The smaller the values ​​of MAE, RMSE and MAPE are, the smaller the model prediction error is and the better the prediction effect of the model is. 2 The closer the value is to 1, the better the model fit is.

[0145] The present invention is a material demand forecasting method based on iWOA-iTransformer. By introducing the improved whale optimization algorithm (iWOA) based on the idea of ​​differential evolution, the types of candidate solutions of the population are increased, and the optimization ability of the whale optimization algorithm is improved. The iTransformer model is optimized by the iWOA algorithm for hyperparameter optimization, so that the prediction effect of the iTransformer model is optimal. Figure 4 , Figure 5 , Figure 6 , Figure 7 As shown in the figure, the present invention uses the Alibaba Cloud infrastructure supply chain inventory management decision dataset as the verification object. Compared with the iTransformer model, the MAE, RMSE, and MAPE error metrics are reduced by 16.97%, 13.45%, and 15.45%, respectively. 2 The goodness of fit index was improved by 1.11%. Through relevant experimental verification, the algorithm proposed in this paper has better fitting effect, higher prediction accuracy and smaller error than other algorithms, which can provide a reference for the material demand forecast of enterprises and has good practical application capabilities.

[0146] Embodiment 2

[0147] The present invention also provides a material demand forecasting system based on the iWOA-iTransformer model, including: an acquisition module, an optimization module, a capture module and a forecasting module;

[0148] The acquisition module is used to collect and preprocess the time series data of material consumption in the production process of equipment manufacturing enterprises to obtain a data set;

[0149] The optimization module is used to introduce the improved whale optimization algorithm iWOA and the iTransformer model, and use the improved whale optimization algorithm iWOA to optimize the hyperparameters of the iTransformer model;

[0150] The capture module is used to build an improved iTransformer model based on the optimal solution of hyperparameters, and to capture the correlation between time series in the data set based on the improved iTransformer model;

[0151] The prediction module is used to predict material demand based on the correlation between time series in the data set by training the improved iTransformer model to obtain the final material demand value.

[0152] In this embodiment, the improved whale optimization algorithm iWOA includes: a prey encirclement stage, a spiral predation stage, a random search stage, and a differential evolution algorithm introduction stage;

[0153] Among them, the process of surrounding the prey stage is expressed as:

[0154] D=|C·X * (t)-X(t)|

[0155] X(t+1)=X * (t)-A·D

[0156] Among them, D is the hunting step length of humpback whales and prey, t is the current iteration number, X * (t) is the position vector of the prey, X(t) is the position vector of the humpback whale, and A and C are coefficient vectors;

[0157] The process of the spiral predation stage is expressed as:

[0158] X(t+1)=D'·e bl ·cos(2πl)+X * (t)

[0159] D'=|X * (t)-X(t)|

[0160] Where D' is the distance between the current individual and the prey, b is the constant defining the logarithmic spiral equation, and l is a random number in the range of [-1,1];

[0161] The process of random search phase is expressed as:

[0162] D=|C·X rand (t)-X(t)|

[0163] X(t+1)=X rand (t)-A·D

[0164] Among them, X rand (t) is the position of a random individual in the group of whales;

[0165] The process of introducing the differential evolution algorithm stage is expressed as:

[0166] Set the mutation factor F and crossover probability CR to get the mutation strategy:

[0167] U i (j) = X * (j)+F·(X r1 (j)-X r2 (j))

[0168] Among them, i is the current iteration number, j is the current whale individual, and U i (j) is the new offspring generated by the j-th whale individual in the i-th iteration, r1 and r2 are two random numbers and satisfy r1≠r2≠i, X * (j) is the most abundant individual in the current population, are different individuals randomly selected from the population, and the parameter j is set rand represents a random number in the range of 1-n, where n is the number of whales in the population, when r≤CR||j=j rand Then a crossover mutation operation is performed to generate new individuals.

[0169] In this embodiment, the process of using the improved whale optimization algorithm iWOA to optimize the hyperparameters of the iTransformer model includes:

[0170] Through differential evolution, the original population is mutated and hybridized to generate new individuals and obtain the optimal solution for the hyperparameters.

[0171] In this embodiment, the improved iTransformer model includes: an embedding layer, a multi-head self-attention layer, a normalization layer, and a feedforward neural network layer;

[0172] Among them, the embedding layer converts the preprocessed data into vectors and implements the embedding operation based on the multi-layer perceptron MLP;

[0173] The multi-head self-attention layer is initialized with a randomly set weight matrix W. Q , W K and W V, the input time series data and the weight matrix are processed into query vector Q, keyword vector K and value vector V in the form of dot product, the query vector Q, keyword vector K and value vector V are projected h times and then the Attention calculation is performed respectively, and the Contact function is used to splice the attention calculation results in the multi-dimensional space, and finally the attention results of all subspaces are connected to obtain the final output result of the multi-head self-attention layer;

[0174] The normalization layer applies normalization to the series representation of individual variables, so that the characteristics of all variables are in a relatively uniform distribution;

[0175] The feedforward neural network layer is implemented using a multi-layer perceptron (MLP), which allows the feedforward network to act on the entire sequence and extract the intrinsic properties of the sequence.

[0176] In this embodiment, based on the improved iTransformer model, the method for capturing the correlation between time series in the data set includes:

[0177] The improved iTransformer model maps the entire time series of variables into the smallest modeling unit - words, and uses the variable as the main body to naturally mine multivariate associations based on words through the attention mechanism;

[0178] The differences caused by different units between variables are eliminated through the cooperation of the feedforward network and the normalization layer.

[0179] The above embodiments are only descriptions of the preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the protection scope determined by the claims of the present invention.

Claims

1. Material demand forecasting method based on iWOA-iTransformer model, It is characterized in that The following steps are involved: Collect and preprocess the time series data of material consumption in the production process of equipment manufacturing enterprises to obtain a data set; Introduce the improved whale optimization algorithm iWOA and iTransformer model, and use the improved whale optimization algorithm iWOA to optimize the hyperparameters of the iTransformer model; An improved iTransformer model is constructed based on the optimal solution of hyperparameters, and the correlation between time series in the data set is captured based on the improved iTransformer model; Based on the correlation between time series in the data set, the material demand is predicted by training the improved iTransformer model to obtain the final material demand value.

2. According to the material demand forecasting method based on the iWOA-iTransformer model of claim 1, It is characterized in that The improved whale optimization algorithm iWOA includes: the prey encirclement stage, the spiral predation stage, the random search stage, and the introduction of the differential evolution algorithm stage; Among them, the process of surrounding the prey stage is expressed as: D=|C·X * (t)-X(t)| X(t+1)=X * (t)-A·D Among them, D is the hunting step length of humpback whales and prey, t is the current iteration number, X * (t) is the position vector of the prey, X(t) is the position vector of the humpback whale, and A and C are coefficient vectors; The process of the spiral predation stage is expressed as: X(t+1)=D'·e bl ·cos(2πl)+X * (t) D'=|X * (t)-X(t)| Where D' is the distance between the current individual and the prey, b is the constant defining the logarithmic spiral equation, and l is a random number in the range of [-1,1]; The process of random search phase is expressed as: D=|C·X rand (t)-X(t)| X(t+1)=X rand (t)-A·D Among them, X rand (t) is the position of a random individual in the group of whales; The process of introducing the differential evolution algorithm stage is expressed as: Set the mutation factor F and crossover probability CR to get the mutation strategy: U i (j)=X * (j)+F·(X r1 (j)-X r2 (j)) Among them, i is the current iteration number, j is the current whale individual, and U i (j) is the new offspring generated by the jth whale individual in the i-th iteration, r 1 and r 2 are two random numbers that satisfy r 1 ≠r 2 ≠i,X * (j) is the most abundant individual in the current population, are different individuals randomly selected from the population, and the parameter j is set rand represents a random number in the range of 1-n, where n is the number of whales in the population, when r≤CR||j=j rand Then a crossover mutation operation is performed to generate new individuals.

3. According to the material demand forecasting method based on the iWOA-iTransformer model of claim 2, It is characterized in that The method of using the improved whale optimization algorithm iWOA to optimize the hyperparameters of the iTransformer model includes: Through differential evolution, the original population is mutated and hybridized to generate new individuals and obtain the optimal solution for the hyperparameters.

4. According to the material demand forecasting method based on the iWOA-iTransformer model of claim 1, It is characterized in that The improved iTransformer model includes: embedding layer, multi-head self-attention layer, normalization layer, and feedforward neural network layer; Among them, the embedding layer converts the preprocessed data into vectors and implements the embedding operation based on the multi-layer perceptron MLP; The multi-head self-attention layer is initialized with a random weight matrix W. Q , W K and W V , the input time series data and the weight matrix are processed into query vector Q, keyword vector K and value vector V in the form of dot product, the query vector Q, keyword vector K and value vector V are projected h times and then the Attention calculation is performed respectively, and the Contact function is used to splice the attention calculation results in the multi-dimensional space, and finally the attention results of all subspaces are connected to obtain the final output result of the multi-head self-attention layer; The normalization layer applies normalization to the series representation of individual variables, so that the characteristics of all variables are in a relatively uniform distribution; The feedforward neural network layer is implemented using a multi-layer perceptron (MLP), which allows the feedforward network to act on the entire sequence and extract the intrinsic properties of the sequence.

5. According to the material demand forecasting method based on the iWOA-iTransformer model of claim 4, It is characterized in that Based on the improved iTransformer model, methods for capturing the association between time series in the data set include: The improved iTransformer model maps the entire time series of variables into the smallest modeling unit - words, and uses the variable as the main body to naturally mine multivariate associations based on words through the attention mechanism; The differences caused by different units between variables are eliminated through the cooperation of the feedforward network and the normalization layer.

6. Material demand forecasting system based on iWOA-iTransformer model, It is characterized in that include: Acquisition module, optimization module, capture module and prediction module; The acquisition module is used to collect and preprocess the time series data of material consumption in the production process of equipment manufacturing enterprises to obtain a data set; The optimization module is used to introduce the improved whale optimization algorithm iWOA and the iTransformer model, and use the improved whale optimization algorithm iWOA to optimize the hyperparameters of the iTransformer model; The capture module is used to build an improved iTransformer model based on the optimal solution of hyperparameters, and to capture the correlation between time series in the data set based on the improved iTransformer model; The prediction module is used to predict material demand based on the correlation between time series in the data set by training the improved iTransformer model to obtain the final material demand value.

7. The material demand forecasting system based on the iWOA-iTransformer model according to claim 6, It is characterized in that The improved whale optimization algorithm iWOA includes: the prey encirclement stage, the spiral predation stage, the random search stage, and the introduction of the differential evolution algorithm stage; Among them, the process of surrounding the prey stage is expressed as: D=|C·X * (t)-X(t)| X(t+1)=X * (t)-A·D Among them, D is the hunting step length of humpback whales and prey, t is the current iteration number, X * (t) is the position vector of the prey, X(t) is the position vector of the humpback whale, and A and C are coefficient vectors; The process of the spiral predation stage is expressed as: X(t+1)=D'·e bl ·cos(2πl)+X * (t) D'=|X * (t)-X(t)| Where D' is the distance between the current individual and the prey, b is the constant defining the logarithmic spiral equation, and l is a random number in the range of [-1,1]; The process of random search phase is expressed as: D=|C·X rand (t)-X(t)| X(t+1)=X rand (t)-A·D Among them, X rand (t) is the position of a random individual in the group of whales; The process of introducing the differential evolution algorithm stage is expressed as: Set the mutation factor F and crossover probability CR to get the mutation strategy: U i (j)=X * (j)+F·(X r1 (j)-X r2 (j)) Among them, i is the current iteration number, j is the current whale individual, and U i (j) is the new offspring generated by the j-th whale individual in the i-th iteration, r1 and r2 are two random numbers and satisfy r1≠r2≠i, X * (j) is the most abundant individual in the current population, are different individuals randomly selected from the population, and the parameter j is set rand represents a random number in the range of 1-n, where n is the number of whales in the population, when r≤CR||j=j rand Then a crossover mutation operation is performed to generate new individuals.

8. The material demand forecasting system based on the iWOA-iTransformer model according to claim 7, It is characterized in that The process of using the improved whale optimization algorithm iWOA to optimize the hyperparameters of the iTransformer model includes: Through differential evolution, the original population is mutated and hybridized to generate new individuals and obtain the optimal solution for the hyperparameters.

9. The material demand forecasting system based on the iWOA-iTransformer model according to claim 6, It is characterized in that The improved iTransformer model includes: embedding layer, multi-head self-attention layer, normalization layer, and feedforward neural network layer; Among them, the embedding layer converts the preprocessed data into vectors and implements the embedding operation based on the multi-layer perceptron MLP; The multi-head self-attention layer is initialized with a random weight matrix W. Q , W K and W V , the input time series data and the weight matrix are processed into query vector Q, keyword vector K and value vector V in the form of dot product, the query vector Q, keyword vector K and value vector V are projected h times and then the Attention calculation is performed respectively, and the Contact function is used to splice the attention calculation results in the multi-dimensional space, and finally the attention results of all subspaces are connected to obtain the final output result of the multi-head self-attention layer; The normalization layer applies normalization to the series representation of individual variables, so that the characteristics of all variables are in a relatively uniform distribution; The feedforward neural network layer is implemented using a multi-layer perceptron (MLP), which allows the feedforward network to act on the entire sequence and extract the intrinsic properties of the sequence.

10. The material demand forecasting system based on the iWOA-iTransformer model according to claim 9, It is characterized in that Based on the improved iTransformer model, methods for capturing the association between time series in the data set include: The improved iTransformer model maps the entire time series of variables into the smallest modeling unit - words, and uses the variable as the main body to naturally mine multivariate associations based on words through the attention mechanism; The differences caused by different units between variables are eliminated through the cooperation of the feedforward network and the normalization layer.

Images