Hybrid CNN Transformer Time Series Forecasting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing time series forecasting methods struggle to effectively model both short-term and long-term temporal dependencies, leading to challenges in predicting future events, particularly in domains like finance where complex interactions need to be understood.
Innovation Solution
The proposed solution involves a hybrid approach using convolutional neural networks (CNNs) and transformer models within a forecasting platform. CNNs are employed to capture local patterns for short-term dependencies, while transformers are used to learn global context and long-term dependencies. The outputs from both models are then processed by a multilayer perceptron classifier to assign a classification, such as a sign prediction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If CNNs are used to model short-term dependencies, then local pattern recognition is improved, but long-term dependency modeling deteriorates
Solution Approach 1:
The patent combines CNN and Transformer models into a hybrid architecture where CNN extracts local temporal features from time series data and Transformer captures global contextual relationships. The CNN component processes input sequences to identify short-term patterns, while the Transformer component processes the extracted features to model long-term dependencies, achieving both local precision and global reliability
2Reliability
If Transformers are used to learn global context, then long-term dependency modeling is improved, but local pattern recognition deteriorates
Solution Approach 1:
The patent segments the time series analysis into two distinct processing stages: first, CNN convolutions extract local temporal patterns and features from contiguous segments of the input sequence; second, Transformer self-attention mechanisms process these extracted features to capture global contextual relationships. This segmentation allows each model to specialize in its strength while working together comprehensively
Data Source
AI summary
In some aspects, the techniques described herein relate to a method including: receiving, at a forecasting platform, a time series; partitioning the time series into a plurality of partitions; processing the time series with a convolutional neural network machine learning model; generating, by the convolutional neural network machine learning model, a plurality of tokens, wherein the plurality of tokens are based on the time series; processing the plurality of tokens with a transformer machine learning model; generating, by the transformer machine learning model, a transformer vector, wherein the transformer vector is based on relationships among the plurality of tokens determined by the transformer machine learning model; and assigning, by a multilayer perceptron classifier, a classification to the transformer vector.