LSTM-SVR subway station temperature prediction method based on multi-period features

Abstract

This invention discloses an LSTM-SVR subway station temperature prediction method based on multi-periodic features. Specifically, the method checks data integrity, analyzes the periodicity and energy density of the amplitude frequency response curve using a discrete Fourier transform, and extracts periods with clear amplitude peaks. Next, a time series decomposition model is used to perform a moving average on the data to remove the largest periodic fluctuations. The Fourier spectrum of the residual is then observed to further remove the periodicity at the clear peaks. Here, an LSTM training model is used for long-term trend data, which has good prediction accuracy for nonlinear features. Periodic feature data is trained using an SVR model, which is fast, has low feature dimensionality, and is less susceptible to overfitting. An improved time series decomposition method is also proposed, which further decomposes the periodic fluctuations to obtain periodic components and residual terms. Other periodic term components are then extracted, and a prediction model for periodic fluctuation factors is then constructed.

Description

[Technical Field]

[0001] The present invention belongs to the field of railway transportation, and relates to the field of environment control, temperature control and temperature prediction of subway stations in railway transportation, and in particular to an LSTM-SVR (long short-term memory network-support vector regression) subway station temperature prediction method based on multi-period features. [Background technology]

[0002] As the most advanced form of urban public transportation, urban rail transport is currently tasked with transporting large numbers of passengers. Subway station environments are characterized by large passenger flows, high population densities, and poor airflow. The temperature and humidity inside subway stations have a significant impact on passenger comfort and satisfaction. Therefore, real-time adjustment of subway cooling units and air conditioning systems based on highly accurate temperature predictions is of great significance for improving passenger experience.

[0003] Regarding the temperature inside subway stations, in natural environments, the most significant factors are undoubtedly the season and weather, followed by the time of day. Therefore, it has a clear time-series character and multi-periodicity. Temperature is also related to factors such as the building area, number of passengers, train operation, ventilation and air conditioning, and the adjustment status of the cooling unit. Time-series forecasting primarily uses large amounts of historical data to train a model and then makes predictions based on current environmental factors, making it suitable for adjusting ventilation and air conditioning systems. Regarding time-series data forecasting, researchers in China and abroad have proposed many forecasting methods, including traditional time series decomposition and autoregressive integrated moving average (ARIMA) models. However, these methods are primarily suited to stationary time series; their performance drops sharply for non-stationary series with large changes in mean, variance, and frequency over time. Other machine learning methods include support vector regression (SVR), recurrent neural networks (RNNs), long-short-term memory networks (LSTMs), echo-state networks (ESNs), and two-stage attention recurrent neural networks (DA-RNNs). As the length and variables of time series data increase, it becomes necessary to improve the real-time nature and accuracy of forecasts through feature engineering, selection of appropriate machine learning algorithms, and techniques such as dimensionality reduction and noise removal. Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention was made in light of the above circumstances, and its objectives are as follows: In response to the situation where temperature data has multiple cycles and fluctuations in each cycle cannot be accurately extracted and forecasted, the present invention analyzes the characteristics of past temperature time series data from subway sensors, and addresses the problem of the low accuracy of single temperature forecasting models by combining the long-term trend, multi-cycle, and irregular fluctuation characteristics of the temperature data itself to develop an LSTM-SVR time series temperature forecasting method based on multi-cycles, which is a combined forecasting model using long-short-term memory networks (LSTM) and support vector regression (SVR) theory based on time series decomposition. [Means for solving the problem]

[0005] The technical solution adopted by the present invention to achieve the above objective is as follows: a LSTM-SVR subway station temperature prediction method based on multi-period features, which performs a discrete Fourier transform on temperature time series data, extracts periodic features that are clearly at peak values ​​based on the peak values ​​of the spectrum, performs a moving average on the period with the largest amplitude to remove the periodicity, performs feature engineering, and further trains an LSTM model to predict long-term trends and cyclical fluctuations. Next, the original time series data is divided by the moving average to obtain a periodic feature sequence, and the remaining periodicity is analyzed based on the Fourier spectrum, and the remaining periodic features are extracted and feature engineering is performed. While training an SVR model, the time series decomposition method is improved to predict the periodicity, and the results are added together to obtain a predicted value. Specifically, the method includes the following steps: Step 1: Separate the eigenvalues ​​of the training set and continuously train the LSTM-SVR model through cross-validation and parameter optimization. In step 2, the improved time series decomposition forecasting model is adopted to construct the periodic fluctuation factors, and the forecast results are added and output.

[0006] Furthermore, step 1 specifically includes: In step S101, a discrete Fourier transform is performed on the temperature time series data to obtain the amplitudes of different frequency components. The discrete Fourier transform formula is:

number

[0007] In S102, each feature component is extracted using a time series decomposition method.

[0008] Time series feature Y t It is divided into four factors using the decomposition method, and the following relationship exists: Y t =f(T t ,St ,C t ,I t ) Long-term trend factors t represents the overall trend that a time series shows over a long duration and is generally determined by basic and intrinsic factors specific to the phenomenon, which cause the phenomenon to fluctuate in a continuous manner along a single direction, rising, falling, or undulating around its previous level.

[0009] Cyclical fluctuation factor S t represents the periodic fluctuations that a time series exhibits as the seasons change due to the influence of natural seasonal factors (weather conditions) or socially customary seasonal factors (holidays).

[0010] Cyclical Fluctuation Factor C t represents a periodic fluctuation in the time series, with alternating rises and falls occurring over a period of several years.

[0011] Random Variation I t represents the irregular fluctuations that the time series exhibits due to the influence of random factors.

[0012] S103, long-term trend factor T t Regarding LSTM training model, i.e., Forget Gate:f t =σ(W f h t-1 +W f X t +B f ) Input gate: i t =σ(W i h t-1 +W i X t +B i ) is adopted to calculate the cell state of the current input based on the previous output and the current input, i.e.,

number

[0013] S104, the cyclical fluctuation factor C t and periodic fluctuation factor S t We will adopt the SVR training model for this. The temperature data of subway stations is highly stationary, and cyclical fluctuation factor C t and periodic fluctuation factor S t Since the nonlinear and periodic features of C are obvious and the dimension is low, the present invention adopts the SVR algorithm to t and St The data model training has high generalization ability and can effectively avoid overfitting problems.The basic idea of ​​the SVR algorithm is to use a kernel function to map linearly inseparable low-dimensional space data into a high-dimensional space to achieve linear regression.Its advantage is that it has a unique advantage when dealing with problems with small data volumes and obvious nonlinear characteristics, and the establishment of the decision function only requires a small number of support vectors, resulting in good robustness.

[0014] Furthermore, step 2 specifically includes: S201, employing an improved time series decomposition prediction model to estimate periodic fluctuation factors S t Build. Conventional time series decomposition methods can only perform seasonal analysis on time series with strong monocyclical characteristics, and for time series with strong polycyclical characteristics, they ignore the strong periodic features of other periods. Therefore, the present invention proposes an improved time series decomposition method, which continuously decomposes periodic fluctuations to obtain periodic components and residual terms, and then extracts other periodic term components, and then constructs a forecast model of periodic fluctuation factors.

[0015] S(t)=f(S1(t),S2(t),S3(t),...,S n (t)) where S1(t), S2(t), S3(t), . . ., S n (t) is a period T1, T2, T3, . . ., T n corresponds to a periodic fluctuation of And T1>T2>T3>……>T n and I t Ignore the influence of (using randomness to add up when predicting),

number

number

[0016] In S202, prediction is made for each item using the improved time series decomposition prediction model and the prediction set data, and the prediction results are added and output. [Effects of the Invention]

[0017] The present invention has the following advantageous effects compared to the prior art. 1. In this invention, LSTM-SVR is used to train each trend feature data on subway ambient temperature time series data to observe whether the data features can be effectively learned, and the prediction results are added to observe the overall effect. A comparative analysis with a single model reveals that the LSTM-SVR combined prediction model method based on multi-period features has significant advantages over a single model, being able to more effectively separate features in longer data series, better train data based on features, and achieving high prediction accuracy for non-stationary, non-linear, and time series with obvious periodicity.

[0018] 2. In addition, the present invention processes each decomposed time series in parallel to improve the training speed, and classifies large changes in some data caused by holidays or unexpected events into uniform residual terms. When making predictions, effective measures are taken if the input data meets external conditions such as holidays or unexpected events, thereby further improving accuracy. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is a diagram showing the time series of the original data in this embodiment. [Figure 2] FIG. 2 is a diagram showing the spectrum after the discrete Fourier transform of the time series of the original data in this example. [Figure 3] FIG. 3 is a diagram illustrating the training of the long short-term memory network LSTM model of this embodiment. [Figure 4] FIG. 4 shows the eigenvalue separation process of the training set and the training of the LSTM-SVR model in this embodiment. [Figure 5] FIG. 5 shows the training and prediction process based on the LSTM-SVR combined model of this embodiment. [Figure 6] FIG. 6 is a diagram showing prediction results by the LSTM model of this embodiment. [Figure 7] FIG. 7 is a diagram showing the prediction results by the SVR model of this embodiment. [Figure 8] FIG. 8 is a diagram showing prediction results by the LSTM-SVR model of this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] In order to allow those skilled in the art to better understand the technical features and technical content of the present invention, reference is made to the following specific embodiments and specification drawings related to the present invention, which are used for reference and explanation without limiting the present invention.

[0021] The data in this example will be described. In this invention, the temperature data of a subway station platform is used as the verification target, and the temperature control of the station building uses control devices such as exhaust fans, cooling units, and ventilation / air conditioning. Data is collected by integrating an environmental monitoring system in the background into a comprehensive monitoring system, with a collection frequency of 10 minutes per data point from 2019 to 2020. 50,142 pieces of temperature data were collected, and after inspection and sorting, incomplete data, data with obvious abnormal changes, and out-of-range data were eliminated, leaving 47,839 valid temperature data points. All drawings used in this invention are generated based on this data.

[0022] Specifically, this embodiment uses an LSTM-SVR subway station temperature prediction method based on multi-period features. Specifically, a discrete Fourier transform is performed on the temperature time series data, and periodic features at the peak values ​​are extracted based on the spectral peaks. A moving average is performed on the period with the largest amplitude to remove the periodicity, and feature engineering is performed. An LSTM model is then trained to predict long-term trends and cyclical fluctuations. The original time series data is then divided by the moving average to obtain a periodic feature sequence. The remaining periodicity is analyzed based on the Fourier spectrum, and the remaining periodic features are extracted and feature engineering is performed. An SVR model is then trained, and the time series decomposition method is improved to predict the periodicity, and the results are added together to obtain a predicted value. Specifically, the method includes the following steps: Step 1: Separate the eigenvalues ​​of the training set and continuously train the LSTM-SVR model through cross-validation and parameter optimization. In step 2, the improved time series decomposition forecasting model is adopted to construct the periodic fluctuation factors, and the forecast results are added and output.

[0023] Furthermore, in S101, a discrete Fourier transform is performed on the temperature time series data to obtain the amplitudes of different frequency components. The discrete Fourier transform formula is:

number

[0024] In S102, each feature component is extracted using a time series decomposition method.

[0025] Time series feature Yt It is divided into four factors using the decomposition method, and the following relationship exists: Y t =f(T t ,S t ,C t ,I t ) Long-term trend factors t represents the overall trend that a time series shows over a long duration and is generally determined by basic and intrinsic factors specific to the phenomenon, which cause the phenomenon to fluctuate in a continuous manner along a single direction, rising, falling, or undulating around its previous level.

[0026] Cyclical fluctuation factor S t represents the periodic fluctuations that a time series exhibits as the seasons change due to the influence of natural seasonal factors (weather conditions) or socially customary seasonal factors (holidays).

[0027] Cyclical Fluctuation Factor C t represents a periodic fluctuation in the time series, with alternating rises and falls occurring over a period of several years.

[0028] Random Variation I t represents the irregular fluctuations that the time series exhibits due to the influence of random factors.

[0029] In the present invention, the time series decomposition method is a multiplicative model, i.e., Y t =T t *S t *C t *I t will be adopted.

[0030] The multiplicative model has the following characteristics: (1)S t is periodic, i.e.,

number

number

number

[0031] S103, long-term trend factor T t For this, we adopt the LSTM training model. In the present invention, the long-term trend factor T t Using LSTM for training can extract nonlinear features more effectively. LSTM (Long Short-Term Memory) is a special type of recurrent neural network (RNN) that is primarily used to solve the gradient vanishing and gradient explosion problems in the training process for longer sequences. Compared to regular RNNs, LSTMs have additional forget gates, input gates, and output gates, allowing them to perform better with longer sequences.

[0032] Here, the operator σ represents the Sigmoid activation function, which outputs a value between 0 and 1, the operator × represents matrix element multiplication, and h t-1 represents the output of the LSTM at the previous time point, and g t-1 represents the state of the LSTM at the previous time, and X t represents the current time input, and f t is the current time of the forget gate X t represents the updated value for i t , g tis the current time of the input gate X t represents the updated value for o t is the current time of the output gate X t represents the updated value for W f , W i , W g , W σ and B f , B i , B g , B σ represent their update weight matrix and offset, respectively. Forget Gate:f t =σ(W f h t-1 +W f X t +B f ) Input gate: i t =σ(W i h t-1 +W i X t +B i ) Calculate the cell state of the current input based on the previous output and the current input, i.e.

number

[0033] S104, the cyclical fluctuation factor C t and periodic fluctuation factor S t We will adopt the SVR training model for this. The temperature data of subway stations is highly stationary, and cyclical fluctuation factor C t and periodic fluctuation factor S t Since the nonlinear and periodic features of C are obvious and the dimension is low, the present invention adopts the SVR algorithm to t and S t The data model training has high generalization ability and can effectively avoid overfitting problems.The basic idea of ​​the SVR algorithm is to use a kernel function to map linearly inseparable low-dimensional space data into a high-dimensional space to achieve linear regression.Its advantage is that it has a unique advantage when dealing with problems with small data volumes and obvious nonlinear characteristics, and the establishment of the decision function only requires a small number of support vectors, resulting in good robustness.

[0034] Given the subway station temperature time series data, the hyperplane f(x)=ω T By dividing x+b, the distance of the sample point x to the decision surface is γ=|ω T x+b| / ||ω||.

[0035] Assume that a maximum error ε is allowed between f(x) and y, and calculate the loss only if the absolute value of the difference between f(x) and y is greater than ε, i.e., |y i -(ω T x i +b)|≦ε. Also, to maximize the margin, we need to add a constraint, γ=2 / ||ω||, as the sum of the distances of the two heterogeneous support vectors to the hyperplane. To maximize the margin, ||ω|| -1 It is good to maximize only ||ω|| 2 This corresponds to minimizing This results in a cost function of

number

number

number

number

number

[0036] The above process of S101 to S104 is the eigenvalue separation of the training set and the LSTM-SVR model training process, the flow of which is shown in Figure 4. S201: Repeatedly perform moving average, long-term change removal, residual determination, and next cycle selection according to the method shown in FIG. 4 to continuously train the LSTM and SVR models, and optimize the LSTM-SVR combined model through cross-validation and parameter optimization.

[0037] S202, employing the improved time series decomposition prediction model to estimate periodic fluctuation factors S t Build. Conventional time series decomposition methods can only perform seasonal analysis on time series with strong monocyclical characteristics, and for time series with strong polycyclical characteristics, they ignore the strong periodic features of other periods. Therefore, the present invention proposes an improved time series decomposition method, which continuously decomposes periodic fluctuations to obtain periodic components and residual terms, and then extracts other periodic term components, and then constructs a forecast model of periodic fluctuation factors.

[0038] S(t)=f(S1(t),S2(t),S3(t),...,S n (t)) where S1(t), S2(t), S3(t), . . ., S n (t) is a period T1, T2, T3, . . ., T n corresponds to a periodic fluctuation of And T1>T2>T3>……>T n and I t Ignore the influence of (using randomness to add up when predicting),

number

number

[0039] In S203, prediction is made for each item using the improved time series decomposition prediction model and the prediction set data, and the prediction results are added and output.

[0040] In the data set used in the verification of this embodiment, up to 40,000 data items were taken as the training set, and the remaining 7,839 data items were taken as the prediction set. A comparative experiment was conducted between a single LSTM, SVR model and an LSTM-SVR combined model based on multi-period features, and three prediction evaluation indices, namely MAE (Mean Absolute Error), RMSE (Root Mean Square Error), and MAPE (Mean Absolute Percentage Error), were selected for evaluation. The comparison of prediction and evaluation indices is shown in Table 1. [Table 1]

[0041] As can be seen from the above, LSTM-SVR was used to train each trend feature data on subway ambient temperature time series data to observe whether it could effectively learn the data features. The prediction results were then added together to observe the overall effect. A comparative analysis with a single model revealed the following: The LSTM-SVR combined prediction model based on multi-period features has significant advantages over single models, being able to more effectively separate features in longer data series, better train data based on features, and achieve high prediction accuracy for non-stationary, nonlinear, and clearly periodic time series. Furthermore, parallel processing of each decomposed time series improves training speed, and significant data fluctuations caused by holidays or unexpected events are classified as uniform residual terms. This allows for effective handling when input data meets external conditions such as holidays or unexpected events, further improving accuracy.

[0042] It should be noted that the above is only a preferred embodiment of the present invention, and those skilled in the art may make some improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also deemed to fall within the protection scope of the present invention.

Claims

1. The comprehensive monitoring system The method is characterized by: performing a discrete Fourier transform on the temperature time series data; extracting periodic features at the peak values ​​based on the peak values ​​of the spectrum; performing a moving average with the period of the maximum amplitude to remove the periodicity; performing feature engineering; further training an LSTM model to predict the long-term trend and cyclical fluctuations; then performing division of the original time series data and the moving average to obtain a periodic feature series; analyzing the remaining periodicity based on the Fourier spectrum; extracting the remaining periodic features; performing feature engineering; training an SVR model; and simultaneously employing a time series decomposition method to predict the periodicity; and adding up the results of the predicted periodicity to obtain a predicted value. The LSTM-SVR subway station temperature prediction method includes: Step 1: separating the eigenvalues ​​of the training set and continuously training the LSTM-SVR model by cross-validation and parameter optimization; Step 2: constructing periodic fluctuation factors by adopting a time series decomposition prediction model, and adding and outputting the prediction results; Specifically, step 2 includes: Step 201: A time series decomposition prediction model is adopted to calculate periodic fluctuation factors S ｔ Build The periodic fluctuation is further decomposed to obtain periodic components and residual terms, and other periodic term components are extracted, and then a prediction model of periodic fluctuation factors is constructed; S(t) = f(S １ (t), S ２ (t), ..., S ｎ (t)) Here, S １ (t), S ２ (t), ..., S ｎ (t) is a period T １ , T ２ , ..., T ｎ and T １ >T ２ …>T ｎ and I ｔ Ignoring the influence of,addition,using randomness when predicting, [Equation 5] and Here, S １ｔ (t) is T １ represents the periodic component of the period, and S １ｒ (t) is T １ represents the residual term of another periodic component based on the periodic component, i.e., S １ｒ (t) = f(S ２ (t), S ３ (t), S ４ (t), ..., S ｎ (t)) and S １ｒ (t) = S ２ｔ (t)(1+S ２ｒ (t)) is satisfied, [Equation 6] Steps to get The LSTM-SVR subway station temperature forecasting method based on multi-period features includes, in step 202, a step of forecasting for each item using an improved time series decomposition forecasting model and forecast set data, and adding and outputting the forecast results.

2. Step 1 Step 101: Perform a discrete Fourier transform on the temperature time series data to obtain the amplitudes of different frequency components thereof, and the discrete Fourier transform formula is: [Equation 1] and where x(n) is a finite length sequence of length N and X(k) is the spectrum corresponding to each point in the sequence; Step 102: Extracting each feature component using a time series decomposition method; Time series feature Y ｔ is divided into four factors using the decomposition method, and the following relationship exists: Y ｔ =f(T ｔ ,S ｔ ,C ｔ ,I ｔ ) Long-term trend factor T ｔ represents the overall trend that a time series shows over a long duration, determined by factors specific to the phenomenon, which cause the phenomenon to fluctuate in a continuous upward or downward direction, or to fluctuate in waves around its previous level, Periodic fluctuation factor S ｔ represents the periodic fluctuations that the time series exhibits as the seasons change due to natural seasonal changes, Cyclical Fluctuation Factor C ｔ represents a periodic fluctuation in the time series, with a cycle of several years, in which increases and decreases alternately appear, Irregular fluctuation factors I ｔ represents the irregular fluctuations exhibited by the time series due to the influence of random factors, In the time series decomposition method, a multiplicative model, i.e., Y t =T t *S t *C t *I t The multiplicative model is S ｔ is periodic, that is, [Equation 2] That is, [Equation 3] And, 【number】 represents the moving average value of the time series data, 【number】 and I ｔ Running a moving average on a variable removes randomness. ｔ are independent random variable sequences that follow a normal distribution with mean 0; Step 103: The long-term trend factor T ｔ We adopted the LSTM training model for The LSTM training model includes a forget gate, an input gate, and an output gate, and can perform well on longer sequences; Forget Gate: f ｔ = σ(W ｆ h ｔ－１ +W ｆ X ｔ +B ｆ ) Input gate: i ｔ = σ(W ｉ h ｔ－１ +W ｉ X ｔ +B ｉ ) Calculate the cell state of the current input based on the previous output and the current input, i.e. [Equation 4] Output gate: o ｔ = σ(W ｏ h ｔ－１ +W ｏ X ｔ +B ｏ ) Output power: h ｔ =o ｔ (tanh(g) ｔ )) where the operator σ represents the Sigmoid activation function, which outputs a value between 0 and 1, the operator × represents matrix element multiplication, and h ｔ－１ represents the output of the LSTM at the previous time, and g ｔ－１ represents the state of the LSTM at the previous time, and X ｔ represents the input of the current time, and f ｔ is the current time of the forget gate X ｔ represents the updated value for i ｔ , g ｔ is the current time of the input gate X ｔ represents the updated value for o ｔ is the current time of the output gate X ｔ represents the updated value for W ｆ , W ｉ , W ｇ , W σ and B ｆ , B ｉ , B ｇ , B σ represent their update weight matrices and offsets, respectively; The LSTM training model has the same feature as the BP neural network, namely backpropagation. It uses the chain rule to find the gradient to each weight, trains samples continuously, and updates the matrix weights using stochastic gradient descent to find the global optimum. Step 104: The cyclical fluctuation factor C ｔ and periodic fluctuation factor S ｔ and employing an SVR training model for predicting temperature of a subway station based on multi-period features according to claim 1 .

3. Specifically, step 104 is as follows: For the given subway station temperature time series data, the hyperplane f(x) = ω Ｔ By dividing x+b, the distance of the sample point x to the decision surface is γ=|ω Ｔ x+b| / ||ω||, Assume that a maximum error ξ is allowed between f(x) and y, and calculate the loss only if the absolute value of the difference between f(x) and y is greater than ξ, i.e., |y ｉ - (ω Ｔ x ｉ + b)|≦ξ, Also, to maximize the margin, a constraint must be added, and the sum of the distances of the two heterogeneous support vectors to the hyperplane is given by γ = 2 / ||ω||, To maximize the margin, ||ω|| -１ It is good to maximize only ||ω|| ２ corresponds to minimizing This results in a cost function of [Equation 7] and where C is a regularization constant and l ξ is a low-sensitivity loss function, [Equation 8] To select an appropriate kernel function so that the training samples are linearly separable in the feature space, we use the slack variables 【number】 Introduced This results in the objective function being [Equation 9] and at the same time, [Equation 10] Fulfilling By introducing Lagrange multipliers, we transform the problem into a dual problem and use the kernel function [0011] When you select SVR, [0012] where, 【number】 The LSTM-SVR subway station temperature prediction method based on multi-period features according to claim 1, characterized in that: is a Lagrange multiplier.

Images