Transform variant-based high-speed train late recovery time prediction method

Abstract

The invention discloses a high-speed train late recovery time prediction method based on Transform variants, and the method comprises the steps: obtaining high-speed train late recovery time sequence data, carrying out the preprocessing of the high-speed train late recovery time sequence data, and obtaining the preprocessed time sequence data; the preprocessed time sequence data are input into a high-speed train late recovery time prediction model, a high-speed train late recovery time prediction result is obtained, and the high-speed train late recovery time prediction model is constructed by connecting an Inverted Transform model and a CNN model. Complex feature representation is automatically learned, nonlinear features in traffic data can be better fitted, spatial-temporal features in late recovery data are effectively captured, train late recovery time prediction precision is improved, and generalization ability of the model to the spatial-temporal format train late recovery data is improved while precision is improved.

Description

Technical Field

[0001] The invention belongs to the field of intelligent transportation technology, and in particular relates to a high-speed train delay recovery time prediction method based on Transformer variants. Background Art

[0002] Train delay recovery prediction is crucial to improving transportation efficiency and passenger satisfaction. Accurate prediction can effectively support dispatcher decision-making and reduce operating costs. At present, the existing methods for train delay recovery prediction can be mainly divided into three categories: statistical prediction methods, intelligent computing prediction methods, and machine learning prediction methods. Statistical prediction methods make inferences and predictions by establishing relationships between variables. There are a series of obvious defects and limitations, such as: uncertainty in data quality, which may cause the model to fail to accurately capture the true situation of delay recovery; many statistical models are based on the assumption that data conforms to a specific distribution, which is often not true in practical applications, resulting in limited applicability of the model; external factors such as emergencies or policy changes are often not included in the model, affecting the accuracy and reliability of the prediction. The train delay recovery method based on intelligent computing mainly relies on a data-based prediction model, using multi-dimensional data such as historical delay records, train running time, and section characteristics to predict train delay recovery. The main limitation of the intelligent computing method is the high requirement for data quality. At the same time, due to differences in network scale and environment, the versatility and portability of the model are limited. In particular, in the absence of sufficient historical data, the reliability of the prediction will be affected. The train delay recovery method based on machine learning predicts the delay recovery process by building a data-driven model. It relies on large-scale historical data and generates high-precision prediction results by mining the complex relationships between variables. Although the prediction method based on machine learning performs well in time series prediction and can effectively capture nonlinear relationships and complex patterns in the data, due to the complexity of the data format, the model still has limitations in capturing spatiotemporal characteristics. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes a high-speed train delay recovery time prediction method based on Transformer variant, which can effectively capture the spatiotemporal characteristics in the data and improve the prediction accuracy.

[0004] To achieve the above object, the present invention provides a high-speed train delay recovery time prediction method based on Transformer variant, comprising:

[0005] Acquire high-speed train delay recovery time series data, preprocess the high-speed train delay recovery time series data, and obtain preprocessed time series data;

[0006] The preprocessed time series data is input into a high-speed train delay recovery time prediction model to obtain a high-speed train delay recovery time prediction result, wherein the high-speed train delay recovery time prediction model is constructed by connecting an InvertedTransformer model and a CNN model.

[0007] Optionally, obtaining preprocessed time series data includes:

[0008] Mean filling processing is performed on the high-speed train delay recovery time series data to obtain pre-processed time series data.

[0009] Optionally, obtaining the high-speed train delay recovery time prediction result includes:

[0010] Input the preprocessed time series data into the Inverted Transformer model to obtain the spatial features in the time series data;

[0011] The spatial features in the time series data are input into the CNN model, the spatial features are further extracted and the processed high-order features are mapped to the output dimension of the task to obtain the high-speed train delay recovery time prediction result.

[0012] Optionally, the Inverted Transformer model consists of an input layer, an embedding layer, a multi-head attention layer, a normalization layer, a feedforward network layer and a linear layer;

[0013] The input layer is used to receive the preprocessed time series data;

[0014] The embedding layer is used to convert the preprocessed time series data into high-dimensional time series data;

[0015] The multi-head attention layer is used to obtain different patterns in the high-dimensional time series data by parallel calculation of multiple attention heads according to the high-dimensional time series data, and to splice the results of each attention head;

[0016] The normalization layer is used to standardize the output of each layer;

[0017] The feedforward network layer is used to perform feature transformation and processing on the concatenated attention results through two layers of fully connected networks;

[0018] The linear layer is used to map the results of feature transformation and processing to the output dimension required by the target task, and obtain the spatial features in the time series data that conform to the output dimension.

[0019] Optionally, the CNN model includes a network architecture of a first parallel branch, a network architecture of a second parallel branch, a network architecture of a third parallel branch, and a network architecture of a fourth parallel branch;

[0020] The network architecture of the first parallel branch, the network architecture of the second parallel branch, and the network architecture of the third parallel branch respectively use convolution kernels of different sizes to further extract spatial features in the time series data;

[0021] The network architecture of the fourth parallel branch enriches feature information of the spatial features in the time series data after further extraction;

[0022] The input features of each branch are concatenated in the channel dimension, and the processed high-dimensional features are mapped to the output dimension of the task.

[0023] Optionally, convolution kernels of different sizes are 1x1, 3x3, and 5x5.

[0024] Optionally, the convolution kernel size of the network architecture of the fourth parallel branch is 3x3.

[0025] Technical effect of the invention: The present invention discloses a high-speed train delay recovery time prediction method based on Transformer variant, which does not require manual intervention for modeling and feature extraction, automatically learns complex feature representation, can better fit the nonlinear characteristics in traffic data, effectively capture the spatiotemporal characteristics in delay recovery data, and improve the prediction accuracy of train delay recovery time. While improving the accuracy, the model's processing ability for train delay recovery data in spatiotemporal format is enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0027] Figure 1 Schematic diagram of the overall network structure of a high-speed train delay recovery time prediction method based on Transformer variants according to an embodiment of the present invention;

[0028] Figure 2 Schematic diagram of the CNN model structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0029] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0030] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0031] like Figure 1 As shown, in this embodiment, a high-speed train delay recovery time prediction method based on a Transformer variant is provided, including: obtaining high-speed train delay recovery time series data, preprocessing the high-speed train delay recovery time series data, and obtaining the preprocessed time series data;

[0032] The preprocessed time series data is input into the high-speed train delay recovery time prediction model to obtain the high-speed train delay recovery time prediction result. The high-speed train delay recovery time prediction model is constructed by connecting the InvertedTransformer model and the CNN model.

[0033] Specifically, the present invention discloses a high-speed train delay recovery time prediction method based on Transformer variant, which does not require manual intervention for modeling and feature extraction, automatically learns complex feature representation, can better fit the nonlinear characteristics in traffic data, effectively capture the spatiotemporal characteristics in delay recovery data, and improve the accuracy of train delay recovery time prediction. While improving the accuracy, the model's processing ability for train delay recovery data in spatiotemporal format is enhanced.

[0034] Furthermore, the preprocessed time series data includes:

[0035] The high-speed train delay recovery time series data is processed by mean filling to obtain the preprocessed time series data.

[0036] Furthermore, the prediction results of high-speed train delay recovery time include:

[0037] Input the preprocessed time series data into the Inverted Transformer model to obtain the spatial features in the time series data;

[0038] The spatial features in the time series data are input into the CNN model, the spatial features are further extracted and the processed high-order features are mapped to the output dimension of the task to obtain the prediction result of the delay recovery time of high-speed trains.

[0039] Specifically, first, the train delay recovery time series data is preprocessed by mean filling and other methods. The processed time series data is used as the input of the Inverted Transformer model. After the processed time series data is input into the Inverted Transformer model, the original numerical data is converted into a high-dimensional vector representation through the embedding layer, laying the foundation for subsequent feature extraction. In the multi-head attention layer, the model captures global dependencies from different time steps through a parallel attention mechanism to ensure that the model can learn complex associations in the time series. Next, the normalization layer standardizes these outputs to maintain data stability and prevent gradient problems in training. Subsequently, the feedforward network layer further transforms the extracted features through two layers of fully connected networks, introduces nonlinear capabilities, and enhances the expressiveness of the model. Finally, the CNN model further extracts spatial features from the time series data and maps the processed high-dimensional features to the output dimensions of the task, generating the final results for time series prediction.

[0040] Furthermore, the Inverted Transformer model consists of an input layer, an embedding layer, a multi-head attention layer, a normalization layer, a feedforward network layer, and a linear layer;

[0041] Input layer, used to receive preprocessed time series data;

[0042] Embedding layer, used to convert preprocessed time series data into high-dimensional time series data;

[0043] The multi-head attention layer is used to obtain different patterns in high-dimensional time series data by calculating multiple attention heads in parallel, and then concatenate the results of each attention head;

[0044] Normalization layer, used to standardize the output of each layer;

[0045] The feed-forward network layer is used to transform and process the concatenated attention results through two layers of fully connected networks;

[0046] The linear layer is used to map the results of feature transformation and processing to the output dimension required by the target task and obtain the spatial features in the time series data that conform to the output dimension.

[0047] The CNN model includes a network architecture of a first parallel branch, a network architecture of a second parallel branch, a network architecture of a third parallel branch, and a network architecture of a fourth parallel branch;

[0048] The network architecture of the first parallel branch, the network architecture of the second parallel branch, and the network architecture of the third parallel branch respectively use convolution kernels of different sizes to further extract spatial features from the time series data;

[0049] The network architecture of the fourth parallel branch enriches the feature information of the spatial features in the further extracted time series data;

[0050] The input features of each branch are concatenated in the channel dimension, and the processed high-dimensional features are mapped to the output dimension of the task.

[0051] Specifically, the Inverted Transformer model consists of an input layer, an embedding layer, a multi-head attention layer, a normalization layer, a feedforward network layer, and a linear layer. The input layer is the starting point of the model and is used to receive raw data. For time series data, it accepts continuous numerical values. The input data is usually high-dimensional and needs to be processed in subsequent layers to extract effective features. The embedding layer converts discrete data into continuous vector representations. For time series data, it converts input features into high-dimensional vectors that are convenient for subsequent processing. The multi-head attention layer enables the model to focus on different parts of the input sequence at the same time by calculating multiple attention heads in parallel. Each head independently calculates the similarity between the query, key, and value to capture different patterns in the sequence, and finally splices the results of each head together for subsequent processing. The role of the normalization layer is to standardize the output of each layer to make the data distribution more stable. It performs normalization independently on each input to ensure that the model maintains the stability of training in the deep structure and prevents the gradient from disappearing or exploding. The feedforward network layer further transforms and processes the output of the multi-head attention layer, introducing more nonlinear capabilities to the model, so that the model can better adapt to complex input data structures. The linear layer is usually part of the output layer, which maps high-dimensional features to the output dimensions required by the target task. For regression tasks, the linear layer directly outputs continuous values. Figure 2 As shown in the figure, the CNN model structure is a network architecture with four parallel branches, each of which extracts multi-scale features through convolution kernels of different sizes. Specifically, the model uses 1x1, 3x3, and 5x5 convolution kernels to extract features of different scales, and further enriches the feature information through a 3x3 maximum pooling branch. Subsequently, the output features of each branch are concatenated in the channel dimension to form the final output. Through this design, the model can simultaneously capture the features of the input signal at different scales, and effectively reduce the amount of calculation through 1x1 convolution, thereby improving the efficiency of the model.

[0052] The present invention realizes the effective capture of spatiotemporal features and the improvement of prediction accuracy through heterogeneous Inverted Transformer model and CNN model. The core advantage of CNN model lies in its powerful spatial feature extraction capability. Through the parallel multi-scale convolution kernel structure, CNN model can simultaneously process fine-grained and coarse-grained spatial features in time series. The Inception structure further enhances the model's ability to capture different scale patterns in local areas, from short-term fluctuations to long-term trend changes, and identifies subtle periodic laws and abnormal fluctuations. In addition, the convolution layer can extract similar patterns in adjacent time steps by utilizing local spatial invariance, making CNN more robust to noise. It can not only improve the feature extraction capability of the model, but also reduce computational overhead. The Inverted Transformer model can capture the global dependencies in the data and the complex patterns of long time series through a powerful self-attention mechanism. The multi-head attention mechanism of the inverted transformer ensures that the model can identify the associations between different time points, making it excellent in processing long-term dependencies and sequential information. The two are combined to make up for the shortcomings of Inverted Transformer in capturing spatial features. CNN is responsible for processing local spatiotemporal features, while Inverted Transformer focuses on capturing global dependencies, achieving effective integration of local and global information, and significantly improving the accuracy and stability of train delay recovery predictions.

[0053] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A high-speed train delay recovery time prediction method based on Transformer variants, characterized in that: include: Acquire high-speed train delay recovery time series data, preprocess the high-speed train delay recovery time series data, and obtain preprocessed time series data; The preprocessed time series data is input into a high-speed train delay recovery time prediction model to obtain a high-speed train delay recovery time prediction result, wherein the high-speed train delay recovery time prediction model is constructed by connecting an InvertedTransformer model and a CNN model.

2. The high-speed train delay recovery time prediction method based on Transformer variant according to claim 1 is characterized in that: The preprocessed time series data includes: Mean filling processing is performed on the high-speed train delay recovery time series data to obtain pre-processed time series data.

3. The high-speed train delay recovery time prediction method based on Transformer variant according to claim 1 is characterized in that: The prediction results of high-speed train delay recovery time include: Input the preprocessed time series data into the Inverted Transformer model to obtain the spatial features in the time series data; The spatial features in the time series data are input into the CNN model, the spatial features are further extracted and the processed high-order features are mapped to the output dimension of the task to obtain the high-speed train delay recovery time prediction result.

4. The high-speed train delay recovery time prediction method based on Transformer variant according to claim 3 is characterized in that: The Inverted Transformer model consists of an input layer, an embedding layer, a multi-head attention layer, a normalization layer, a feedforward network layer, and a linear layer; The input layer is used to receive the preprocessed time series data; The embedding layer is used to convert the preprocessed time series data into high-dimensional time series data; The multi-head attention layer is used to obtain different patterns in the high-dimensional time series data by parallel calculation of multiple attention heads according to the high-dimensional time series data, and to splice the results of each attention head; The normalization layer is used to standardize the output of each layer; The feedforward network layer is used to perform feature transformation and processing on the concatenated attention results through two layers of fully connected networks; The linear layer is used to map the results of feature transformation and processing to the output dimension required by the target task, and obtain the spatial features in the time series data that conform to the output dimension.

5. The high-speed train delay recovery time prediction method based on Transformer variant according to claim 3 is characterized in that: The CNN model includes a network architecture of a first parallel branch, a network architecture of a second parallel branch, a network architecture of a third parallel branch, and a network architecture of a fourth parallel branch; The network architecture of the first parallel branch, the network architecture of the second parallel branch, and the network architecture of the third parallel branch respectively use convolution kernels of different sizes to further extract spatial features in the time series data; The network architecture of the fourth parallel branch enriches feature information of the spatial features in the time series data after further extraction; The input features of each branch are concatenated in the channel dimension, and the processed high-dimensional features are mapped to the output dimension of the task.

6. The high-speed train delay recovery time prediction method based on Transformer variant according to claim 5 is characterized in that: The convolution kernels of different sizes are 1x1, 3x3, and 5x5.

7. The high-speed train delay recovery time prediction method based on Transformer variant according to claim 5 is characterized in that: The convolution kernel size of the network architecture of the fourth parallel branch is 3x3.

Images