Tourism demand forecasting method based on STL and LSTM with attention mechanism

Through the LSTM model of STL decomposition and attention mechanism, the overfitting problem caused by limited data in tourism demand prediction is solved, the prediction accuracy and model simplicity are improved, and more accurate tourism demand prediction is achieved.

CN114091768BActive Publication Date: 2024-11-22ANHUI UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202111414339.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-25
Publication Date
2024-11-22
Estimated Expiration
2041-11-25

AI Technical Summary

Technical Problem

Existing tourism demand forecasting models lead to overfitting problems due to limited data volume, especially in complex modeling data diversity and volume, and there are irrelevant or redundant explanatory variables in the feature selection process, resulting in predictions. Low accuracy.

Method used

Using the tourism demand prediction method based on STL and LSTM with attention mechanism, the time series is decomposed into seasonal sequences, trend sequences and residual sequences through the STL decomposition model, and the attention layer is used to obtain the weights of different features, which will be weighted. The feature input to the LSTM model for prediction, set the training set and test set ratio to 8:2, obtain the final prediction result through the fully connected unit, and optimize the model parameters through grid search.

Benefits of technology

It effectively alleviates the overfitting problem, forms a relatively simple prediction process, improves the appropriateness of feature variable selection and time step length, and significantly improves the accuracy of tourism demand prediction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114091768B_ABST
    Figure CN114091768B_ABST
Patent Text Reader

Abstract

A tourism demand forecasting method based on STL and LSTM with attention mechanism, comprising: obtaining an initial time series from tourist flow data; decomposing the initial time series using an STL decomposition model to obtain three decomposition sequences; inputting the three decomposition sequences into a shared attention layer respectively, obtaining the weights of different features in each sequence, and then using the features with weights as inputs to the LSTM; training the three decomposition sequences using an LSTM model with the same parameters, and forecasting their test sets respectively; the three forecast results enter a fully connected unit; and the three forecast results are added together to obtain the final forecast result of the arrival amount of tourists and output it. The advantages of the present invention are: firstly, using STL to solve the overfitting problem in a highly complex model caused by limited data in the process of tourism demand forecasting, and forming a relatively simple forecasting process; secondly, using Attention‑LSTM to effectively select feature variables and appropriate time steps.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of tourism demand forecasting, and in particular to a tourism demand forecasting method SAL based on LSTM with an attention mechanism combined with seasonal trend decomposition (STL). Background Art

[0002] Tourism planners need accurate forecasts of tourist arrivals. Similarly, destinations and all participants in the tourism value chain, such as transportation departments, tour operators, accommodation providers, event organizers, and retailers, need accurate forecasts to make short-term decisions to meet their business needs, analyze market trends, and develop long-term strategies. Therefore, tourism demand forecasting has always been a research hotspot. Tourism demand forecasting research can be divided into two methods: qualitative and quantitative. Among them, qualitative methods often use people's knowledge, experience, and analytical judgment ability based on the historical data and intuitive materials that have been mastered to make judgments on the nature and degree of future development trends of things, and then use a certain form to synthesize all aspects of the judgment to draw a unified forecast conclusion. However, these methods often have poor generalization ability. Therefore, researchers have been studying quantitative methods to estimate the quantitative relationship between different observations in tourism data. Based on past factor data and passenger flow data, the constructed model can be used to predict future tourist arrivals. Generally speaking, quantitative methods use two strategies to improve performance. The first strategy attempts to incorporate more relevant factors that may affect tourists' travel motivation, while the second strategy is to adopt more complex models with better generalization ability for future trends.

[0003] The existing tourism demand forecasting methods and models can be divided into three categories: time series models, econometric models, and artificial intelligence models. Among them, the time series model uses historical data to predict future tourism demand. For example, the prediction model of ARIMA and its variables has been widely adopted. Goh (Modeling and forecasting tourism demand for arrivals with stochastic nonstationary seasonality and intervention. Tourism management, 2002) et al. proposed the SARIMA method, which captures the seasonality within the univariate time series. In addition, in the early tourism demand forecasting research, exponential smoothing (ETS) and Navie method were proposed. The econometric model uses the relationship between tourism demand and explanatory variables to forecast tourism demand. Traditional econometric models are based on regression methods, such as ordinary least squares (OLS), autoregressive distributed lag model (ADLM), vector autoregression model (VAR), and error correction model, which have been widely used. In recent years, in order to improve the performance of tourism demand forecasting, a mixed data sampling model using Google search data has been proposed. However, the econometric model has certain limitations in the feature selection process. Especially in tourism demand forecasting, how to effectively establish a model with a large number of variables remains a difficult problem. Models based on artificial intelligence have also been successfully used for tourism demand forecasting. Wong (Bayesian models for tourism demand forecasting. Tourism Management, 2006) et al. proposed a Bayesian network model to predict the demand for tourism in Hong Kong and achieved certain results. Cankurt (Tourism demand forecasting using ensembles of regression trees. In: 2016 IEEE 8th International Conference on Intelligent Systems (IS), 2016) used regression trees to predict the demand for tourism in Turkey. As a nonlinear forecasting model, artificial neural networks (ANN) have also been used for tourism forecasting. In recent years, Law (Tourism demand forecasting: A deep learning approach. Annals of tourism research, 2019) et al. introduced deep learning models into tourism demand forecasting and found that deep learning models have strong generalization ability and high accuracy.

[0004] In the process of using AI-based models for forecasting, overfitting is one of the main challenges faced by quantitative forecasting models. In tourism demand forecasting, the quality is affected by two reasons for overfitting. First, in complex modeling, the diversity and volume of data are usually limited. For example, most related studies can only obtain historical data from 5-10 years ago. Wu (Asparse gaussian process regression model for tourism demand forecasting in hong kong. Expert Systems with Applications, 2012) et al. mentioned that ANN often has overfitting problems. Claveria (Tourism demand forecasting with neural network models: different ways of treating information. International Journal of Tourism Research, 2015) et al. believe that multi-layer neural networks require cross-validation to overcome the overfitting problem. Second, irrelevant or redundant explanatory variables may be introduced in the model construction process. Li (Forecasting tourism demand with composite search index. Tourism management, 2017) et al. believe that when using search index to predict tourism demand, excessive search queries will lead to overfitting. Sun et al. (Forecasting tourist arrivals with machine learning and internet search index. Tourism Management, 2019) believe that research on tourism demand forecasting related to search index should deeply consider and avoid overfitting problems. In addition, how to effectively use the limited amount of data to select explanatory variables remains to be explored. In summary, although AI-based methods offer a variety of opportunities, several methodological issues remain unresolved. A literature review shows that existing deep learning models for tourism demand forecasting are often highly complex. The root cause of these problems lies in the limited amount of data and the requirement for additional explanatory variables. Therefore, this study aims to address these deficiencies in order to improve the modeling accuracy of AI-based tourism demand forecasting. Summary of the invention

[0005] The technical problem to be solved by the present invention is how to solve the overfitting problem in a highly complex model caused by limited data volume in the process of tourism demand forecasting.

[0006] The present invention solves the above technical problems through the following technical means: a tourism demand prediction method based on STL and LSTM with attention mechanism, comprising the following steps:

[0007] Step 1: Obtain the number of tourist arrivals within the statistical period from the tourist flow data to obtain the initial time series;

[0008] Step 2: Use the STL decomposition model to decompose the initial time series in step 1 to obtain three decomposition sequences;

[0009] Step 3: Take the season sequence, trend sequence and residual sequence obtained in step 2 as input, input them into a shared attention layer, obtain the weights of different features in each sequence, and then use the weighted features as the input of LSTM;

[0010] Step 4: Establish an LSTM prediction model and set the ratio of the training set and the test set of the three decomposition sequences to 8:2; respectively, use the LSTM model with attention mechanism with the same parameters to train the three decomposition sequences obtained in step 2, and respectively predict their test sets;

[0011] Step 5: The three prediction results obtained in step 4 enter the fully connected unit;

[0012] Step 6: Add these three prediction results to get the final prediction result of the number of tourists

[0013] Step 7: Output prediction results

[0014] As a further optimized technical solution, the following steps are also included:

[0015] The SAL model is trained by minimizing the difference between the actual tourist arrivals and the predicted tourist arrivals. The SAL model includes an STL decomposition model and an LSTM model with an attention mechanism. During the training process, grid search is used to fine-tune the parameters to determine the optimal parameters of the SAL model. In the experiment, the time step is set to 12, the dimension of the input feature is 2725, the number of LSTM neurons is in the range of {16, 32, 64, 128, 256, 512}, the number of fully connected units is in the range of {8, 16, 32, 128, 256}, the dropout_rate is in the range of {0.2, 0.3, 0.5, 0.7, 0.8}, the batch range is {2, 3, 7, 11, 12, 15}, and the iterative test range is {300, 500, 700, 1000, 3000}.

[0016] As a further optimized technical solution, step 2 specifically includes the following steps:

[0017] Step 21: Detrending, assuming T t k is the trend component at the end of the kth iteration in the inner loop, using the original sequence Y t Subtract the trend component T of the previous iteration result t k , that is, Y t ′=Y t -T t k , when k = 0, the initial value T t 0 =0;

[0018] Step 22: Periodic subsequence smoothing. Each periodic subsequence in the detrended sequence is smoothed by local weighted regression to obtain a temporary seasonal sequence, denoted as

[0019] Step 23: Low-pass filtering of the smoothed periodic subsequence. Apply a low-pass filter to This filter contains a moving average of length 12, a moving average of length 3, and then performs local weighted regression to obtain the resulting sequence

[0020] Step 24: Remove the trend of the smoothed periodic subsequence, and the seasonal component of the (k+1)th iteration is Subtract The purpose is to prevent low-frequency information from entering the seasonal term;

[0021] Step 25: Remove seasonal terms, original sequence Y t By subtracting the seasonal component We get a deseasonalized series, the deseasonalized series is

[0022] Step 26: Trend smoothing, deseasonalizing the series Y obtained in step 25 t After local weighted regression smoothing, the trend component T in the (k+1)th iteration is obtained. t k+1 ;

[0023] Step 27: Determine seasonal components and trend component T t k+1 Whether it converges, if it converges, output the seasonal component and trend component T t k+1 , otherwise return to step 21;

[0024] Step 28: Calculate the remaining components

[0025] As a further optimized technical solution, step 3 specifically includes the following steps:

[0026] Step 31: Take the seasonal series, trend series and residual series as input respectively;

[0027] Step 32: Input to a fully connected layer;

[0028] Step 33: Then use the softmax activation function for normalization to obtain the weights of different features in each sequence:

[0029]

[0030] Among them, W is the weight matrix and b is the bias term;

[0031] Step 34: Multiply each feature by the corresponding weight to obtain a weighted feature:

[0032]

[0033] Step 35: Output the weighted features as input to LSTM.

[0034] As a further optimized technical solution, the step 4 is specifically as follows:

[0035] Will As input, LSTM encodes it into hidden state At time step t, h t The hidden state of the LSTM unit output gate is composed of the previous hidden state h t-1 , the current cell state c t and the current input x′ t Updated, the update process is as follows:

[0036] f t =σ(W f ×x′ t +U f ×h t-1 +b f )

[0037] i t =σ(W i ×x′ t +U i ×h t-1 +b i )

[0038]

[0039]

[0040] O t =σ(W o ×x′ t +U o ×h t-1 +b o )

[0041] h t =O t ×tanh(C t )

[0042] Among them, W f , U f , W i , U i , W c , U c , W o and U o is the weight matrix, b f 、b c 、b i and b o is the bias term, tanh is the activation function, and × represents element-by-element multiplication. The output of the last step is sent to a linear regression layer to obtain the prediction results of the seasonal series, trend series, and remaining series respectively.

[0043] The present invention also provides a tourism demand prediction device based on STL and LSTM with attention mechanism, comprising the following modules:

[0044] Initial time series acquisition module: used to obtain the number of tourist arrivals within the statistical period from tourist flow data, so as to obtain the initial time series;

[0045] Time series decomposition module: used to decompose the initial time series in step 1 using the STL decomposition model to obtain three decomposition sequences;

[0046] Attention layer module: The seasonal sequence, trend sequence and residual sequence output by the time series decomposition module are input into a shared attention layer to obtain the weights of different features in each sequence, and then the features with weights are used as the input of the LSTM prediction model;

[0047] LSTM prediction model training module: Set the ratio of the training set and the test set of the three decomposition sequences to 8:2; Use the LSTM model with attention mechanism with the same parameters to train the three decomposition sequences output by the time series decomposition module, and predict their test sets respectively;

[0048] Fully connected unit enters the module: 3 prediction results enter the fully connected unit;

[0049] Addition module: Add these three prediction results to get the final prediction result of the number of tourists

[0050] Output module: output prediction results

[0051] As a further optimized technical solution, the following modules are also included:

[0052] The training module is used to train the SAL model by minimizing the difference between the actual tourist arrivals and the predicted tourist arrivals. The SAL model includes an STL decomposition model and an LSTM model with an attention mechanism. During the training process, grid search is used to fine-tune the parameters to determine the optimal parameters of the SAL model. In the experiment, the time step is set to 12, the dimension of the input feature is 2725, the number of LSTM neurons is in the range of {16, 32, 64, 128, 256, 512}, the number of fully connected units is in the range of {8, 16, 32, 128, 256}, the dropout_rate is in the range of {0.2, 0.3, 0.5, 0.7, 0.8}, the batch range is {2, 3, 7, 11, 12, 15}, and the iterative test range is {300, 500, 700, 1000, 3000}.

[0053] As a further optimized technical solution, the time series decomposition module specifically includes:

[0054] Detrending unit: Assume T t k is the trend component at the end of the kth iteration in the inner loop, using the original sequence Y t Subtract the trend component T of the previous iteration result t k , that is, Y′ t =Y t -T t k , when k = 0, the initial value T t 0 =0;

[0055] Periodic subsequence smoothing unit: Each periodic subsequence in the detrended sequence is smoothed by local weighted regression to obtain a temporary seasonal sequence, denoted as

[0056] Low-pass filtering unit for smoothing periodic subsequences: A low-pass filter is applied to This filter contains a moving average of length 12, a moving average of length 3, and then performs local weighted regression to obtain the resulting sequence

[0057] Remove the trend unit of the smooth periodic subsequence: The seasonal component of the (k+1)th iteration is Subtract

[0058] Season-removing unit: original sequence Y t By subtracting seasonal components We get a deseasonalized series, the deseasonalized series is

[0059] Trend smoothing unit: the deseasonalized series Y obtained in the deseasonalization unit t After local weighted regression smoothing, the trend component T in the (k+1)th iteration is obtained. t k+1 ;

[0060] Convergence judgment unit: judging seasonal components and trend component T t k+1 Whether it converges, if it converges, output the seasonal component and trend component T t k+1 , otherwise return to the detrended unit;

[0061] Residual component calculation unit: calculate the residual component

[0062] As a further optimized technical solution, the attention layer module specifically includes:

[0063] Input unit: takes the seasonal series, trend series and residual series as input respectively;

[0064] Fully connected layer unit: input to a fully connected layer;

[0065] Normalization unit: Then use the softmax activation function for normalization to obtain the weights of different features in each sequence:

[0066]

[0067] Among them, W is the weight matrix and b is the bias term;

[0068] Feature acquisition unit: multiply each feature by the corresponding weight to obtain weighted features:

[0069]

[0070] Feature output unit: Outputs weighted features as input to the LSTM prediction model training module.

[0071] As a further optimized technical solution, the LSTM prediction model training module specifically performs the following method:

[0072] Will As input, LSTM encodes it into hidden state At time step t, h t The hidden state of the LSTM unit output gate is composed of the previous hidden state h t-1 , the current cell state c t and the current input x t 'Updated, the update process is as follows:

[0073] f t =σ(W f ×x′ t +U f ×h t-1 +b f )

[0074] i t =σ(W i ×x′ t +U i ×h t-1 +b i )

[0075]

[0076]

[0077] O t =σ(W o ×x′ t +U o ×h t-1 +b o )

[0078] h t =O t ×tanh(C t )

[0079] Among them, W f , U f , W i , U i , W c , U c , W o and U o is the weight matrix, b f 、b c 、b i and b ois the bias term, tanh is the activation function, and × represents element-by-element multiplication. The output of the last step is sent to a linear regression layer to obtain the prediction results of the seasonal series, trend series, and remaining series respectively.

[0080] The advantages of the present invention are:

[0081] Firstly, STL is used to alleviate the overfitting problem in highly complex models caused by limited data in the tourism demand forecasting process, and a relatively simple forecasting process is formed;

[0082] Secondly, Attention-LSTM is used to effectively select feature variables and appropriate time steps. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 is a SAL tourism demand forecasting model diagram provided by an embodiment of the present invention;

[0084] Figure 2 It is a six-step diagram of STL provided by an embodiment of the present invention;

[0085] Figure 3 It is a structural diagram of an LSTM unit provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0086] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0087] First, the SAL model is introduced. In this paper, the SAL model is introduced to alleviate the overfitting problem in tourism demand forecasting. The SAL model includes STL and Attention-LSTM (LSTM with attention mechanism). Among them, STL simplifies the prediction method. The previous STL model decomposes the input data into 3 different sequences, and does not distinguish the importance of different influencing factors in the sequence. Different influencing factors also contribute differently to the prediction of tourist arrivals. Therefore, in order to highlight the role of different influencing factors, this paper uses LSTM based on attention mechanism to obtain more accurate feature variables. In this section, the combination of STL and Attention-LSTM is explained in detail in the tourism demand forecasting problem.

[0088] The time series data of tourist arrivals at T time steps is represented by the vector Y T ={y1,y2,...,yT}. Therefore, the corresponding predicted time series data Y T The eigenvector of can be expressed as: T ={x1,x2,...,x T}, where T is the length of the time series. For a time step of t, the input variable with n exogenous features can be defined as: In time step t, y t represents the number of tourists, x t Represents various factors, such as corresponding Search Intensity Index (SII) data from search engines. Multi-factors of length T and the number of tourists arriving at time step T As the input of the prediction function F, it is used to predict the number of tourist arrivals in the future δ time steps The formula is as follows:

[0089]

[0090] Where δ is the time step to be predicted in the future. Then, the tourist arrivals in the future δ time steps are predicted using the existing data of T time steps and the corresponding factors. Considering the nonlinearity and complexity of tourism demand prediction, the function F must capture the nonlinear mapping relationship between multivariate factors and target tourism arrivals.

[0091] Figure 1 This is the SAL tourism demand prediction model. First, the tourist arrival data is preprocessed; then, the preprocessed data is decomposed by STL; finally, the Attention-LSTM model is used for prediction.

[0092] Based on the SAL model, the functions of each module in the model are described in detail as follows:

[0093] STL is a classic time series decomposition method proposed by Cleveland RB. It plays an extremely important role in time series data analysis, especially in the case of strong seasonality. STL is significantly different from traditional decomposition methods, providing more robust components to decompose time series with outliers. STL decomposes tourism data into three parts: trend, season, and residual. For example, within time step t, the number of tourists reaching Y t After STL decomposition, it can be considered as trend T t , Season S t and remainder R t The sum of the three components is as follows:

[0094] Y t =T t +S t +R t

[0095] The importance of different influencing factors in a sequence is different, so in order to make the feature representation of different sequences more accurate, it is necessary to distinguish the importance of different influencing factors. Therefore, this paper introduces an attention layer to obtain the weights of different influencing factors in the sequence.

[0096] The LSTM network is an extension of the recurrent neural network (RNN), which solves the gradient vanishing problem of the RNN and allows long-term storage of information. It can capture nonlinear trends in the data and remember previous information for a long time. Therefore, LSTM has been successfully applied to deal with time series problems. The purpose of tourism demand forecasting is to predict the current number of tourist arrivals based on previous multivariate factors. Therefore, the data is decomposed by STL and enters the attention layer. Then, after the weight distribution of the attention layer, the weighted features are input into the LSTM for prediction.

[0097] The detailed steps are as follows:

[0098] Step 1: Obtain the number of tourist arrivals within the statistical period from Macau tourist flow data to obtain the initial time series;

[0099] Step 2: Use the STL decomposition model to decompose the initial time series in step 1 to obtain three decomposition sequences;

[0100] Step 3: Take the season sequence, trend sequence and remaining sequence obtained in step 2 as input, input them into a shared attention layer, obtain the weights of different features in each sequence, and then use the weighted features as the input of LSTM.

[0101] Step 4: Establish an LSTM prediction model and set the ratio of the training set and the test set of the three decomposition sequences to 8:2; respectively, use the LSTM model with attention mechanism with the same parameters to train the three decomposition sequences obtained in step 2, and respectively predict their test sets;

[0102] Step 5: The three prediction results obtained in step 4 enter the fully connected unit (Dense layer);

[0103] Step 6: Add these three prediction results to get the final prediction result of the number of tourists

[0104] Step 7: Output prediction results

[0105] The above prediction method also includes the following steps:

[0106] The SAL model is trained by minimizing the difference between the actual tourist arrivals and the predicted tourist arrivals. During the training process, grid search is used to fine-tune the parameters to determine the optimal parameters of the SAL model. In the experiment, the time step is set to 12 and the dimension of the input feature is 2725. The number of LSTM neurons ranges from {16, 32, 64, 128, 256, 512}, and the number of fully connected units ranges from {8, 16, 32, 128, 256}. The dropout_rate ranges from {0.2, 0.3, 0.5, 0.7, 0.8}, the batch ranges from {2, 3, 7, 11, 12, 15}, and the iterative test ranges from {300, 500, 700, 1000, 3000}.

[0107] The step 2 specifically includes the following steps:

[0108] Step 21: Detrending. Assume that T t k is the trend component at the end of the kth iteration in the inner loop, using the original sequence Y t Subtract the trend component T of the previous iteration result t k , that is, Y′ t =Y t -T t k , when k = 0, the initial value T t 0 =0;

[0109] Step 22: Smoothing of periodic subsequences. Each periodic subsequence in the detrended sequence is smoothed by local weighted regression to obtain a temporary seasonal sequence, denoted as

[0110] Step 23: Low-pass filtering of the smoothed periodic subsequence. Apply a low-pass filter to This filter contains a moving average of length 12, a moving average of length 3, and then performs local weighted regression to obtain the resulting sequence

[0111] Step 24: Remove the trend of the smoothed periodic subsequence. The (k+1)th iteration seasonal component is Subtract The purpose is to prevent low-frequency information from entering the seasonal term.

[0112] Step 25: Remove seasonal terms. Original sequence Y t By subtracting seasonal components We get a deseasonalized series, the deseasonalized series is

[0113] Step 26: Trend smoothing. The deseasonalized series Y obtained in step 25 is t After local weighted regression smoothing, the trend component T in the (k+1)th iteration is obtained. t k+1 .

[0114] Step 27: Determine seasonal components and trend component T t k+1 Whether it converges, if it converges, output the seasonal component and trend component T t k+1 , otherwise return to step 21.

[0115] Step 28: Calculate the remaining components

[0116] The step 3 specifically includes the following steps:

[0117] Step 31: Take the seasonal series, trend series and residual series as input respectively;

[0118] Step 32: Input to a fully connected layer (Dense layer);

[0119] Step 33: Then use the softmax activation function for normalization to obtain the weights of different features in each sequence:

[0120]

[0121] Among them, W is the weight matrix and b is the bias term.

[0122] Step 34: Multiply each feature by the corresponding weight to obtain a weighted feature:

[0123]

[0124] Step 35: Output the weighted features as input to LSTM.

[0125] Described step 4 is specifically as follows:

[0126] Will As input, LSTM encodes it into hidden state At time step t, h t The hidden state of the LSTM unit output gate is composed of the previous hidden state h t-1 , the current cell state c t and the current input x′ t Updated. The update process is as follows:

[0127] ft =σ(W f ×x′ t +U f ×h t-1 +b f )

[0128] i t =σ(W i ×x′ t +U i ×h t-1 +b i )

[0129]

[0130]

[0131] O t =σ(W o ×x′ t +U o ×h t-1 +b o )

[0132] h t =O t ×tanh(C t )

[0133] Among them, W f , U f , W i , U i , W c , U c , W o and U o is the weight matrix, b f 、b c 、b i and b o is the bias term, tanh is the activation function, and × represents element-by-element multiplication. The output of the last step is sent to a linear regression layer to obtain the prediction results of the seasonal series, trend series, and residual series respectively.

[0134] In the tourism demand forecasting problem, the study of time series mainly relies on historical tourism data to predict future trends. Time series data can be decomposed into several components and then predicted separately. However, many decomposition methods, such as wavelet transform or empirical mode decomposition, require complex reasoning processes or fuzzy modes. To alleviate these problems, we adopt the seasonal trend decomposition method (STL) and combine STL with LSTM with attention mechanism. Unlike previous studies, their main purpose is to decompose historical data into three sequences and regard different influencing factors in each sequence as equally important. In fact, different influencing factors contribute differently to the sequence representation. Therefore, in order to obtain a more accurate sequence representation, this paper uses the attention mechanism to obtain the weights of different influencing factors, and then uses the weighted sequence as the input of LSTM to obtain more accurate prediction results.

[0135] This paper verifies the performance of the proposed SAL model based on the Macau tourism demand dataset Macau2018. The dataset consists of two parts: monthly tourist arrival data and SII data. SII refers to data related to Macau's tourism industry, such as several keywords in six main categories, namely catering, transportation, accommodation, tourism, shopping and entertainment. Table 1 gives some selected seed search keywords.

[0136] Table 1 Search keywords for Macau tourism seeds

[0137]

[0138] The walk-forward validation method is used to compare with other state-of-the-art methods to evaluate the performance of the tourism demand forecasting model proposed in this paper. The walk-forward validation method is used to simulate a real-world forecasting scenario where new tourist arrival data is available at the end of the month and is used for the next month. During the forecasting process, the time step window of the training data is first set to 12 months, and then one month is added to each time step to forecast the next month.

[0139] To verify the performance of the proposed tourism demand forecasting model, we used a Naive method, a support vector regression (SVR) model, an artificial neural network (ANN), an ARIMA model and DLM as the benchmark model. The SAL model used the Trend and Remainder sequences in the experiment, while the benchmark model used the original tourist arrivals. Based on the data of the previous n months To predict the next month. The first dimension of the DLM input is the total number of months for training, followed by the time step dimension, and then the factor vector created by the SII data and the tourist arrivals of the corresponding month. Naive uses the tourist arrivals 12 months ago as The SVR and ANN models use the data from the past 12 months as input and predict The ANN model has a hidden layer with an activation function and is trained using the back-propagation algorithm. ARIMA uses AR order (p, q) times difference to obtain a stationary sequence and MA order q, trains on tourist data from the past 12 months, and predicts the tourist arrivals for the next 12 months, which is incremented during the traversal validation process. Since SVR and ANN are not efficient in processing data with a large number of features, the maximum mutual information coefficient (MIC) is used to filter out features with a low correlation with tourist arrivals.

[0140] This article selects three common model evaluation indicators, root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE), to measure the prediction performance. The formula is as follows:

[0141]

[0142]

[0143]

[0144] The model with the lowest median value of these indicators is considered the best prediction model. To ensure the robustness of the tourism demand prediction model, we conducted five walk-forward validations on all prediction models and predicted the passenger flow for 12 months from 2013 to 2017 (5 years).

[0145] For the prediction of tourist arrivals, Tables 2, 3 and 4 summarize the comparison of the prediction results of tourist arrivals using the proposed method and five models, namely DLM, SVR, ANN, ARIMA and Naive, in terms of MAPE, MAE and RMSE.

[0146] Table 2 Comparison of 5-year forecast results of MAPE index

[0147]

[0148] From the analysis in Table 2, we can see that under the MAPE index, the prediction accuracy of the SAL prediction model proposed in this paper and the DLM prediction model without STL dropped from 3.460% to 1.631%, an increase of 1.829%. This result shows that the DLM prediction model using STL has better prediction performance and can improve the prediction accuracy. Compared with SVR, ANN, ARIMA, and Naive, the prediction values ​​dropped from 6.268%, 13.936%, 6.411%, and 5.231% to 1.631%, respectively, and the prediction accuracy increased by 4.637%, 12.305%, 4.780%, and 3.600%, respectively. Therefore, the SAL prediction model proposed in this paper has the best prediction performance compared with the five prediction models under the MAPE index.

[0149] Table 3 Comparison of five-year forecast results of MAE indicators

[0150]

[0151] According to the prediction results shown in Table 3, under the MAE indicator, the SAL prediction model proposed in this paper has a better average than the prediction model that was not adopted, and its prediction results are also the best compared with other comparison models. This reflects that the SAL model has better prediction performance. Therefore, the SAL prediction model proves its prediction accuracy under the MAE indicator.

[0152] Table 4 Comparison of RMSE index prediction results for 5 years

[0153]

[0154] From the comparison of the RMSE index prediction results for 5 years shown in Table 4, it can be found that the mean of the SAL model is the smallest, followed by the mean of the DLM model. Among them, the mean of Naive is close to the mean of DLM, which shows that the prediction performance of the DLM prediction result is less improved than that of the initial Naive model. The difference between the SAL prediction model and the Naive mean is larger, which shows that the SAL model can improve the prediction accuracy well. Among them, the model mean is the largest, which shows that the ANN model has the worst prediction performance under the RMSE index. Compared with other comparison models, the SAL model has the smallest mean, which reflects its best prediction performance.

[0155] The above experimental results show that compared with the other five prediction models, the SAL model proposed in this paper has obvious advantages in prediction accuracy under the three indicators of MAPE, MAE and RMSE. In addition, the SAL model effectively alleviates the overfitting problem in tourism demand forecasting and produces a lower average error rate than other models.

[0156] In summary, this paper proposes a new tourism demand forecasting model SAL, which uses the STL decomposition method to solve the overfitting problem caused by limited data, alleviates data overfitting, and improves the accuracy of tourism demand forecasting. The new forecasting model is compared with DLM, SVR, ANN, ARIMA and Naive models to verify the robustness of the new model and confirm its better performance in tourism demand forecasting.

[0157] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A tourism demand forecasting method based on STL and LSTM with attention mechanism, characterized in that: The steps include: Step 1: Obtain the number of tourist arrivals within the statistical period from the tourist flow data to obtain the initial time series; Step 2: Use the STL decomposition model to decompose the initial time series in step 1 to obtain three decomposition sequences; Step 3: Take the season sequence, trend sequence and residual sequence obtained in step 2 as input, input them into a shared attention layer, obtain the weights of different features in each sequence, and then use the weighted features as the input of LSTM; Step 4: Establish an LSTM prediction model and set the ratio of the training set and the test set of the three decomposition sequences to 8:2; respectively, use the LSTM model with attention mechanism with the same parameters to train the three decomposition sequences obtained in step 2, and respectively predict their test sets; Step 5: The three prediction results obtained in step 4 enter the fully connected unit; Step 6: Add these three prediction results to get the final prediction result of the number of tourists Step 7: Output prediction results The step 2 specifically includes the following steps: Step 21: Detrending, assuming T t k is the trend component at the end of the kth iteration in the inner loop, using the original sequence Y t Subtract the trend component T of the previous iteration result t k , that is, Y t ′=Y t -T t k , when k = 0, the initial value T t 0 =0; Step 22: Periodic subsequence smoothing. Each periodic subsequence in the detrended sequence is smoothed by local weighted regression to obtain a temporary seasonal sequence, denoted as Step 23: Low-pass filtering of the smoothed periodic subsequence. Apply a low-pass filter to This filter contains a moving average of length 12, a moving average of length 3, and then performs local weighted regression to obtain the resulting sequence Step 24: Remove the trend of the smoothed periodic subsequence, and the seasonal component of the (k+1)th iteration is Subtract The purpose is to prevent low-frequency information from entering the seasonal term; Step 25: Remove seasonal terms, original sequence Y t By subtracting seasonal components We get a deseasonalized series, the deseasonalized series is Step 26: Trend smoothing, deseasonalizing the series Y obtained in step 25 t After local weighted regression smoothing, the trend component T in the (k+1)th iteration is obtained. t k+1 ; Step 27: Determine seasonal components and trend component T t k+1 Whether it converges, if it converges, output the seasonal component and trend component T t k+1 , otherwise return to step 21; Step 28: Calculate the remaining components 2. A tourism demand forecasting method based on STL and LSTM with attention mechanism as claimed in claim 1, characterized in that: The following steps are also included: The SAL model is trained by minimizing the difference between the actual tourist arrivals and the predicted tourist arrivals. The SAL model includes an STL decomposition model and an LSTM model with an attention mechanism. During the training process, grid search is used to fine-tune the parameters to determine the optimal parameters of the SAL model. In the experiment, the time step is set to 12, the dimension of the input feature is 2725, the number of LSTM neurons is in the range of {16, 32, 64, 128, 256, 512}, the number of fully connected units is in the range of {8, 16, 32, 128, 256}, the dropout_rate is in the range of {0.2, 0.3, 0.5, 0.7, 0.8}, the batch range is {2, 3, 7, 11, 12, 15}, and the iterative test range is {300, 500, 700, 1000, 3000}.

3. A tourism demand forecasting method based on STL and LSTM with attention mechanism as claimed in claim 1, characterized in that: The step 3 specifically comprises the following steps: Step 31: Take the seasonal series, trend series and residual series as input respectively; Step 32: Input to a fully connected layer; Step 33: Then use the softmax activation function for normalization to obtain the weights of different features in each sequence: Among them, W is the weight matrix and b is the bias term; Step 34: Multiply each feature by the corresponding weight to obtain a weighted feature: Step 35: Output the weighted features as input to LSTM.

4. A tourism demand forecasting method based on STL and LSTM with attention mechanism as claimed in claim 1, characterized in that: Described step 4 is specifically as follows: Will As input, LSTM encodes it into hidden state At time step t, h t The hidden state of the LSTM unit output gate is composed of the previous hidden state h t-1 , the current cell state c t and the current input x t 'Updated, the update process is as follows: f t =σ(W f ×x t ′+U f ×h t-1 +b f ) I t =σ(W i ×x t ′+U i ×h t-1 +b i ) The t =σ(W o ×x t ′+U o ×h t-1 +b o ) h t =O t ×tanh(C t ) Among them, W f , U f , W i , U i , W c , U c , W o and U o is the weight matrix, b f , b c , b i and b o is the bias term, tanh is the activation function, and × represents element-by-element multiplication. The output of the last step is sent to a linear regression layer to obtain the prediction results of the seasonal series, trend series, and remaining series respectively.

5. A tourism demand prediction device based on STL and LSTM with attention mechanism, characterized in that: Includes the following modules: Initial time series acquisition module: used to obtain the number of tourist arrivals within the statistical period from tourist flow data, so as to obtain the initial time series; Time series decomposition module: used to decompose the initial time series in step 1 using the STL decomposition model to obtain three decomposition sequences; Attention layer module: The seasonal sequence, trend sequence and residual sequence output by the time series decomposition module are input into a shared attention layer to obtain the weights of different features in each sequence, and then the features with weights are used as the input of the LSTM prediction model; LSTM prediction model training module: Set the ratio of the training set and the test set of the three decomposition sequences to 8:2; Use the LSTM model with attention mechanism with the same parameters to train the three decomposition sequences output by the time series decomposition module, and predict their test sets respectively; Fully connected unit enters the module: 3 prediction results enter the fully connected unit; Addition module: Add these three prediction results to get the final prediction result of the number of tourists Output module: output prediction results The time series decomposition module specifically includes: Detrending unit: Assume T t k is the trend component at the end of the kth iteration in the inner loop, using the original sequence Y t Subtract the trend component T of the previous iteration result t k , that is, Y t ′=Y t -T t k , when k = 0, the initial value T t 0 =0; Periodic subsequence smoothing unit: Each periodic subsequence in the detrended sequence is smoothed by local weighted regression to obtain a temporary seasonal sequence, denoted as Low-pass filtering unit for smoothing periodic subsequences: A low-pass filter is applied to This filter contains a moving average of length 12, a moving average of length 3, and then performs local weighted regression to obtain the resulting sequence Remove the trend unit of the smooth periodic subsequence: The seasonal component of the (k+1)th iteration is Subtract Season-removing unit: original sequence Y t By subtracting seasonal components We get a deseasonalized series, the deseasonalized series is Trend smoothing unit: the deseasonalized series Y obtained in the deseasonalization unit t After local weighted regression smoothing, the trend component T in the (k+1)th iteration is obtained. t k+1 ; Convergence judgment unit: judging seasonal components and trend component T t k+1 Whether it converges, if it converges, output the seasonal component and trend component T t k+1 , otherwise return to the detrended unit; Residual component calculation unit: calculate the residual component 6. A tourism demand forecasting device based on STL and LSTM with attention mechanism as claimed in claim 5, characterized in that: Also includes the following modules: The training module is used to train the SAL model by minimizing the difference between the actual tourist arrivals and the predicted tourist arrivals. The SAL model includes an STL decomposition model and an LSTM model with an attention mechanism. During the training process, grid search is used to fine-tune the parameters to determine the optimal parameters of the SAL model. In the experiment, the time step is set to 12, the dimension of the input feature is 2725, the number of LSTM neurons is in the range of {16, 32, 64, 128, 256, 512}, the number of fully connected units is in the range of {8, 16, 32, 128, 256}, the dropout_rate is in the range of {0.2, 0.3, 0.5, 0.7, 0.8}, the batch range is {2, 3, 7, 11, 12, 15}, and the iterative test range is {300, 500, 700, 1000, 3000}.

7. A tourism demand forecasting method based on STL and LSTM with attention mechanism as claimed in claim 5, characterized in that: The attention layer module specifically includes: Input unit: takes the seasonal series, trend series and residual series as input respectively; Fully connected layer unit: input to a fully connected layer; Normalization unit: Then use the softmax activation function for normalization to obtain the weights of different features in each sequence: Among them, W is the weight matrix and b is the bias term; Feature acquisition unit: multiply each feature by the corresponding weight to obtain weighted features: Feature output unit: Outputs weighted features as input to the LSTM prediction model training module.

8. A tourism demand forecasting method based on STL and LSTM with attention mechanism as claimed in claim 5, characterized in that: The LSTM prediction model training module specifically performs the following method: Will As input, LSTM encodes it into hidden state At time step t, h t The hidden state of the LSTM unit output gate is composed of the previous hidden state h t-1 , the current cell state c t and the current input x t 'Updated, the update process is as follows: f t =σ(W f ×x t ′+U f ×h t-1 +b f ) I t =σ(W i ×x t ′+U i ×h t-1 +b i ) The t =σ(W o ×x t ′+U o ×h t-1 +b o ) h t =O t ×tanh(C t ) Among them, W f , U f , W i , U i , W c , U c , W o and U o is the weight matrix, b f , b c , b i and b o is the bias term, tanh is the activation function, and × represents element-by-element multiplication. The output of the last step is sent to a linear regression layer to obtain the prediction results of the seasonal series, trend series, and remaining series respectively.

Citation Information

Cited By

  • LSTM-based data processing method and smart tourism big data processing system

    CN116720546A