Functional Neural Network for Time Series Dimension Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for time series data analysis, such as PCA, AE, LSTM, and FLM, fail to efficiently reduce the number of features and timepoints while capturing temporal information, leading to inaccurate results with complex real-world data, especially in high-frequency scenarios.
Innovation Solution
Bi-Functional Autoencoder (BFAE) using a functional neural network (FNN) for two-way dimension reduction and multi-class classification, employing functional encoders and decoders with continuous neurons and basis expansion to capture non-linear relationships, reducing both features and timepoints while preserving temporal information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If PCA or AE is used for dimension reduction, then the number of features is reduced, but the number of timepoints cannot be reduced and temporal information is not efficiently captured
Solution Approach 1:
The patent segments the dimension reduction process into two independent operations: feature dimension reduction through functional basis expansion and timepoint dimension reduction through temporal smoothing. This segmentation allows each component to address specific aspects of the problem without compromising the other, enabling reduction of both features and timepoints while preserving temporal information.
Solution Approach 2:
The patent introduces functional basis expansion as an additional dimensional transformation layer. By representing time series data in a functional basis space (e.g., Fourier, wavelet, or polynomial bases) before applying dimension reduction, the method transforms the problem from direct feature compression to functional space compression, thereby preserving temporal structures while reducing dimensionality.
2Adaptability or versatility
If FLM is used for classification, then binary classification is achieved, but it is restricted to linear relationships and cannot handle multi-class classification
Solution Approach 1:
The patent extends the functional linear model to achieve multi-functionality: it can perform both binary and multi-class classification tasks. By incorporating a softmax output layer and cross-entropy loss function, the model universally handles different classification scenarios while maintaining the interpretability and efficiency of functional linear modeling.
Solution Approach 2:
The patent changes the output layer parameters from simple sigmoid activation (binary classification) to softmax activation with cross-entropy loss (multi-class classification). This parameter change enables the model to handle multiple classes while preserving the functional linear relationship in the hidden layers, thus maintaining both versatility and accuracy.
3Productivity
If FPCA is used for dimension reduction, then linear representations are provided, but complex relationships cannot be captured
Solution Approach 1:
The patent introduces a functional basis expansion layer as an intermediary between the input data and the dimension reduction process. This intermediary transforms the raw time series data into a functional representation that captures non-linear relationships through basis functions (e.g., polynomial, wavelet, or Fourier bases), while the subsequent linear dimension reduction operates efficiently on this transformed space.
4Quantity of substance
If FAE is used for dimension reduction, then a scalar representation is provided, but it cannot capture rich temporal information
Solution Approach 1:
The patent implements a dynamic functional autoencoder architecture where the functional basis expansion and temporal smoothing operations adapt to capture time-dependent patterns. Unlike static scalar representations, the model dynamically adjusts the functional basis coefficients to preserve temporal structures, enabling dimension reduction while maintaining rich temporal information through the functional representation.
Data Source
AI summary
Systems and methods described herein extend Functional Neural Networks (FNNs) for time series dimension reduction and multi-class classification. Using functional encoders and decoders, the Bi-Functional Autoencoder (BFAE) reduces both the number of features and timepoints (two way) using basis expansion. FNN is also extended to facilitate time series multi-class classification, which enables detecting more than two classes in the data. The functional encoder uses the continuous neurons in the continuous hidden layer to derive a low-dimension latent representation of the data. This representation is then processed by functional decoder to reconstruct the original information. For multi-class classification, the system utilizes cross-entropy loss and a softmax activation function to effectively handle more than two classes to improve classification performance.