Time sequence prediction method based on multilayer perceptron and convolutional neural network
Through the channel independent mechanism combined with multi-layer perceptron and convolutional neural network, the long-term and short-term dependencies of time series are handled respectively, which solves the problem of difficulty in balancing accuracy and efficiency in the existing technology, and achieves high-precision and efficient time series prediction.
Patent Information
- Application Number
- CN202510275794.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-10
- Publication Date
- 2025-08-01
AI Technical Summary
The prior art is difficult to effectively capture long-term and short-term dependencies in time series simultaneously, especially in high-dimensional data and complex dynamic environments, where there is a balance between accuracy and efficiency.
The channel-independent mechanism is used to embed time series data, and the multi-layer perceptron and convolutional neural networks are combined to process long-term and short-term dependencies respectively. Long-term dependency features are extracted through nonlinear transformation and average pooling operations through multi-layer perceptrons, short-term dependency feature extraction is performed using convolutional neural networks, and finally feature fusion and prediction are performed in multi-layer perceptrons.
It realizes comprehensive feature capture of time series, reduces prediction errors, improves calculation efficiency and accuracy, and is suitable for large-scale real-time prediction tasks.
Smart Images
Figure CN120408059A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of time series prediction, and more specifically relates to a time series prediction method based on a multi-layer perceptron and a convolutional neural network. Background Art
[0002] Time series prediction is a process of inferring future trends based on historical time series data, which is currently widely used in many fields such as finance, economy, climate change, and energy management. At present, many methods have been proposed for time series prediction. Some methods are based on the autoregressive integrated moving average (ARIMA) model, which models the time series through bidirectional ARIMA, generates bidirectional prediction values, and obtains the final prediction result by weighted combining these prediction values. Another type of method uses the long short-term memory (LSTM) network, which captures the dependencies in the time series through LSTM and uses the Dropout layer to randomly discard some information to enhance the generalization ability of the model. Then, through the combination of bidirectional LSTM and fully connected layers, integration and non-linear transformation are performed, and finally the prediction result is output. In addition, as an important deep learning model, the convolutional neural network (CNN) was initially mainly applied to image processing, but with its local perception characteristics and parameter sharing mechanism, it has shown great potential in time series prediction. Based on this, researchers have proposed a variety of time series prediction methods based on CNN. Generally, these methods extract features from time series data through a convolutional neural network and then combine with other models for learning to achieve the prediction goal. Some studies have adopted a causal dilated convolutional neural network to construct a prediction model by stacking causal dilated convolutional modules and fully connected layers, etc.
[0003] Although these methods have achieved remarkable results in dealing with time dependencies, there are still certain limitations in effectively extracting both short-term and long-term dependencies simultaneously, especially in high-dimensional data and complex dynamic environments, where it is often difficult to balance the modeling requirements of the two. In addition, these methods usually face a trade-off between accuracy and efficiency, and it is difficult to maintain high computational efficiency while ensuring high prediction accuracy.
[0004] The prior art fails to effectively capture both long-term and short-term dependencies in time series, and there is a balance problem between accuracy and efficiency when dealing with complex time dependencies. Summary of the Invention
[0005] The present invention first performs an embedding representation on the input time series data and adopts a channel-independent mechanism. This mechanism treats each variable as an independent channel and processes it separately through a dedicated encoding path, thereby retaining the unique characteristics of each variable and effectively mining complex patterns in high-dimensional time series data. The channel-independent mechanism ensures the independence of each channel and avoids interference between variables.
[0006] In order to achieve the above object, the present invention is implemented by adopting the following technical solutions: the solution includes:
[0007] Embed the input time series data and process each channel separately using a channel-independent mechanism;
[0008] The embedded time series is input into the multi-layer perceptron (MLP) for nonlinear transformation to extract preliminary time features, and the features are reduced in dimension through average pooling operation;
[0009] Concatenate the pooled features with the embedded time series to generate a new time series;
[0010] Input the new time series into the multi-layer perceptron (MLP) again to obtain the final long-term dependency features;
[0011] The embedded time series is subjected to feature extraction through a convolutional neural network (CNN); the extracted features are input into a multi-layer perceptron (MLP) for final prediction calculation.
[0012] In one embodiment, the embedding representation is performed through a linear layer, which can effectively preserve the unique characteristics of each variable and better mine potential complex patterns in high-dimensional time series data.
[0013] In one embodiment, the average pooling operation is used to reduce the dimension of multi-layer perceptron (MLP) features, which helps to preserve long-term dependency characteristics in time series.
[0014] In one scheme, a new time series is generated by concatenating the pooled features with the embedded time series, while preserving both short-term and long-term dependency information.
[0015] In one approach, feature extraction is performed on the embedded time series using a convolutional neural network (CNN) using multiple one-dimensional convolutional layers.
[0016] In one scheme, each one-dimensional convolutional layer uses a convolution kernel of size 3 to perform a sliding window operation, which helps to extract short-term dependency features.
[0017] In one scheme, the extracted features are processed through nonlinear activation functions and maximum pooling layers to enhance the nonlinear expression ability of the model and reduce the feature dimension.
[0018] In one approach, short-term dependency features extracted by a convolutional neural network (CNN) are fused with long-term dependency features processed by a multi-layer perceptron (MLP). The fused features are then fed into the MLP for final prediction, taking into account both short-term and long-term feature information.
[0019] Advantages of the present invention:
[0020] By separately processing the long-term and short-term dependencies in the time series, the present invention can comprehensively capture its multi-dimensional features. Long-term dependency modeling is used to capture the macro trends in the time series, while short-term dependency modeling focuses on capturing local fluctuations, thus effectively reducing the prediction error. In addition, the present invention combines the convolutional neural network (CNN) with the multi-layer perceptron (MLP) to achieve parallel processing of short-term and long-term dependencies, significantly improving the computational efficiency. While ensuring high-precision prediction, this method optimizes the computational speed and training time, and is particularly suitable for real-time prediction tasks of large-scale time series data.
[0021] The present invention can be widely applied to time series prediction in multiple fields such as finance, meteorology, and industrial production. By improving the prediction accuracy and efficiency, this method can provide more accurate decision-making support for various practical applications, helping enterprises and research institutions better predict future development trends. Brief Description of the Drawings
[0022] Figure 1 is the flowchart of the method of the present invention;
[0023] Figure 2 is the process diagram of the embedding representation;
[0024] Figure 3 is the process diagram of feature extraction of the multi-layer perceptron (MLP) and the pooling operation;
[0025] Figure 4 is the process diagram of feature extraction of the convolutional neural network (CNN). Detailed Embodiments
[0026] To facilitate the understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant drawings. The typical embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.
[0027] Unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as understood by those skilled in the technical field to which the present invention belongs. The terms used in the description of the present invention in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. To facilitate the understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant drawings. The typical embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.
[0028] As Figure 1 shown, the present invention first performs an embedding representation on the input time series data X = {x1, x2, x3, …, x t}. First, we perform an embedding representation on the input high-dimensional time series. This is achieved by adopting a channel-independent mechanism, treating each variable in the time series as an independent channel, and separately processing each channel through a dedicated encoding path. The features of each channel are transformed through a dedicated linear layer, converting the original time series features into a dense vector to form an embedding vector.
[0029] Adopt a channel-independent mechanism that treats each variable in the time series as an independent channel and separately processes each channel through a dedicated encoding path. The time series contains n variables, and each variable x i after embedding is processed through a different encoding path. The features of each channel are obtained through independent mapping to get the embedding representation X e = Embedding(X), where Embedding is a linear layer. The channel-independent mechanism effectively preserves the unique characteristics of each variable and can better mine the potential complex patterns in the high-dimensional time series data.
[0030] To capture the long-term dependencies in the time series, the present invention adopts a multi-layer perceptron (MLP) and a pooling operation. The embedded time series is input into the multi-layer perceptron (MLP) for non-linear transformation to extract preliminary time features. Then, the features are dimensionally reduced through an average pooling operation, retaining the long-term trend information in the time series. The pooled features are concatenated with the original time series to generate a new time series that fuses the long-term dependency information. The fused features are input into the multi-layer perceptron (MLP) again for processing, adjusting them to the same dimension as the original input sequence to form the final long-term dependency features.
[0031] First, input the embedded time series X e into the multi-layer perceptron (MLP) for non-linear transformation to extract preliminary time features. The features after passing through the MLP are H mlp = MLP(X e ). Then, these features are dimensionally reduced through an average pooling operation, retaining the long-term trend information in the time series. The features after average pooling are:
[0032] H pool = AveragePooling(H mlp )
[0033] where AveragePooling represents the average pooling operation, and the pooled features Hpool It can effectively extract long-term trend information.
[0034] Then, the pooled feature H pool is concatenated with the embedded time series X e to generate a new time series X new , which fuses long-term dependence information. The concatenated sequence is:
[0035] X new = Concat(X e , H pool )
[0036] where Concat represents the concatenation operation.
[0037] Finally, the fused feature X new will be input into a multi-layer perceptron (MLP) again to adjust the dimension and ensure consistency with the initial input sequence, obtaining the final long-term dependence feature H long = MLP(X new )
[0038] For the short-term dependence relationship in the time series, the embedded time series is processed by a convolutional neural network (CNN) to extract short-term dependence information. In this step, the time series will be processed through multiple one-dimensional convolutional layers with a convolutional kernel of size 3. In each layer, the output of the convolutional operation will be processed by a non-linear activation function to extract key features. Then, a max-pooling operation is performed to reduce the feature dimension and retain key information.
[0039] The present invention uses a convolutional neural network (CNN) for feature extraction. The embedded time series X e will be processed through multiple one-dimensional convolutional layers. Each convolutional layer uses a convolutional kernel of size 3 for a sliding window operation to extract short-term dependence features. The formula for the convolutional operation is:
[0040] H conv = Conv1D(X e , W)
[0041] where W represents the convolutional kernel.
[0042] After each layer of convolutional operation, the generated feature map will be processed by a non-linear activation function to enhance the non-linear expression ability of the model. The formula for the activation operation is:
[0043] H activation = ReLU(H conv )
[0044] To reduce the feature dimension and retain key information, the feature map is downsampled through a max pooling layer. The formula for the max pooling operation is:
[0045] H pool = MaxPooling(H activation )
[0046] This operation not only reduces the computational complexity but also improves the robustness of the model.
[0047] To enhance the feature extraction ability, the present invention connects three convolutional layers into a deep feature extraction network, where the output of the previous layer serves as the input of the next layer, finally forming an integrated feature representation. The process is expressed by the following formula:
[0048] H short = CNN(CNN(CNN(X e ,W),W),W)
[0049] Among them, the long-term dependency modeling and the short-term dependency modeling are carried out in parallel. During the processing of the model, the long-term dependency and the short-term dependency are independently extracted through a multi-layer perceptron (MLP) and a convolutional neural network (CNN) respectively without interfering with each other. Finally, the short-term dependency feature H short extracted by the convolutional neural network and the long-term dependency feature H long processed by the multi-layer perceptron will be fused. The fused feature H fused is:
[0050] H fused = Concat(H long ,H short )
[0051] Finally, the fused feature is input into a multi-layer perceptron (MLP) for final prediction calculation to obtain the prediction result Y: The short-term dependency feature extracted by the convolutional neural network and the long-term dependency feature processed by the multi-layer perceptron are fused to form a new feature set containing the long-term and short-term dependency information of the time series. Then these features are input into the multi-layer perceptron for final prediction calculation to output the prediction result.
[0052] Y = MLP(H fused )
[0053] Embodiment
[0054] Data input and embedding representation: The present invention first performs an embedding representation on the input time series data and adopts a channel-independent mechanism to ensure the independence of each channel and avoid interference between variables. Figure 2 The embedding representation process is shown as follows:
[0055] Long-term Dependence Modeling: The time series data after embedding representation is input into a multi-layer perceptron (MLP) for non-linear transformation to extract preliminary time features. Then, the features are dimensionally reduced through average pooling operation, and the pooled features retain the long-term trend information in the time series. The pooled features are concatenated with the original time series to obtain a new time series, which integrates long-term dependence information, thus improving the ability to capture long-term trends. Finally, the fused features are input into the multi-layer perceptron (MLP) again to adjust the dimensions to match the original input sequence. Figure 3 Shows the feature extraction process using a multi-layer perceptron (MLP) and pooling operation:
[0056] Short-term Dependence Modeling: The input time series data is subjected to feature extraction through a convolutional neural network (CNN). The time series data passes through a one-dimensional convolutional layer, and each convolutional layer uses a convolutional kernel of size 3 for sliding window operation. The feature maps after each layer of convolution are processed by a non-linear activation function to enhance the non-linear ability of the model. Finally, downsampling is performed through max pooling operation to reduce computational complexity and improve the robustness of the model. Figure 3 Shows the feature extraction process using a convolutional neural network (CNN):
[0057] Feature Fusion and Prediction: Finally, the short-term dependence features extracted by the convolutional neural network are fused with the long-term dependence features extracted by the multi-layer perceptron, and then input into the multi-layer perceptron for final prediction calculation. This process effectively integrates short-term and long-term dependence information, ensuring high-precision prediction results.
[0058] The present invention first performs an embedding representation on the input time series data and adopts a channel-independent mechanism. This mechanism treats each variable as an independent channel and processes it separately through a dedicated encoding path, thereby retaining the unique characteristics of each variable and effectively mining complex patterns in high-dimensional time series data. The channel-independent mechanism ensures the independence of each channel and avoids interference between variables.
[0059] To capture the long-term dependence relationship in the time series, the present invention combines a multi-layer perceptron (MLP) with a pooling operation for feature extraction. First, the embedded time series data is input into the multi-layer perceptron (MLP) for non-linear transformation to extract preliminary time features. Then, the features are dimensionally reduced using average pooling operation to retain the long-term trend information.
[0060] After generating the pooled features, to enhance the modeling ability of long-term dependencies, the pooled features are concatenated with the original time series to obtain a new time series that incorporates long-term dependency information, thereby improving the ability to capture long-term trends. Finally, the fused features are input into a multi-layer perceptron (MLP) again to adjust the dimensions to match the original input sequence.
[0061] For modeling short-term dependencies, the present invention uses a convolutional neural network (CNN) for feature extraction. The input time series data is processed through a one-dimensional convolutional layer, and each convolutional layer uses a convolutional kernel of size 3 for a sliding window operation to extract short-term dependency features. The feature map after each convolution is processed by a non-linear activation function to enhance the non-linear ability of the model.
[0062] To reduce the feature dimensions and retain key features, the feature map is downsampled through a max-pooling layer to reduce the computational complexity and improve the robustness of the model. To further enhance feature extraction, the present invention stacks multiple convolutional layers into a deep feature extraction network, where the output of each layer serves as the input of the next layer, finally forming an integrated feature representation to more comprehensively describe the short-term dynamic characteristics in the time series.
[0063] Finally, the short-term dependency features extracted by the convolutional neural network are fused with the long-term dependency features extracted by the multi-layer perceptron, and then input into the multi-layer perceptron for final prediction calculation. This process effectively integrates short-term and long-term dependency information, ensuring high-precision prediction results and improving the operating efficiency of the model.
[0064] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.
[0065] It should be understood that the detailed description of the technical solutions of the present invention with the aid of the preferred embodiments above is illustrative rather than restrictive. Those of ordinary skill in the art can modify the technical solutions described in each embodiment based on reading the specification of the present invention, or perform equivalent substitution on some of the technical features; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.
Claims
1. A time series prediction method based on a multi-layer perceptron and a convolutional neural network, characterized in that: The described method includes: Performing an embedding representation on the input time series data and separately processing each channel using a channel-independent mechanism; Inputting the embedded time series into a multi-layer perceptron (MLP) for non-linear transformation, extracting preliminary time features, and reducing the dimensionality of the features through an average pooling operation; Concatenating the pooled features with the embedded time series to generate a new time series; Inputting the new time series into the multi-layer perceptron (MLP) again to obtain the final long-term dependence features; Performing feature extraction on the embedded time series through a convolutional neural network (CNN); Inputting the extracted features into a multi-layer perceptron (MLP) for final prediction calculation.
2. The time series prediction method based on a multi-layer perceptron and a convolutional neural network according to claim 1, characterized in that: The described embedding representation is performed through a linear layer, which can effectively retain the unique characteristics of each variable and better mine the potential complex patterns in high-dimensional time series data.
3. A time series prediction method based on a multi-layer perceptron and a convolutional neural network according to claim 1, characterized in that: The process of reducing the dimensionality of the multi-layer perceptron (MLP) features by the average pooling operation helps to retain the long-term dependence characteristics in the time series.
4. A time series prediction method based on a multi-layer perceptron and a convolutional neural network according to claim 1, characterized in that, By concatenating the pooled features with the embedded time series, a new time series is generated, while retaining both short-term and long-term dependence information.
5. A time series prediction method based on a multi-layer perceptron and a convolutional neural network according to claim 1, characterized in that, Feature extraction is performed on the embedded time series through a convolutional neural network (CNN), and multiple one-dimensional convolutional layers are used.
6. A time series prediction method based on a multi-layer perceptron and a convolutional neural network according to claim 5, characterized in that, Each one-dimensional convolutional layer performs a sliding window operation using a convolutional kernel of size 3, which helps to extract short-term dependence features.
7. A time series prediction method based on a multi-layer perceptron and a convolutional neural network according to claim 1, characterized in that The extracted features are processed through a non-linear activation function and a max pooling layer to enhance the non-linear expression ability of the model and reduce the feature dimensionality.
8. A time series prediction method based on a multi-layer perceptron and a convolutional neural network according to claim 1, characterized in that, The short-term dependence features extracted through the convolutional neural network (CNN) and the long-term dependence features processed through the multi-layer perceptron (MLP) are fused; The fused features are input into the multi-layer perceptron (MLP) for final prediction calculation, taking into account both short-term and long-term feature information.