Neural Network Embedding for Sparse Date Feature Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for forecasting time series data face challenges due to the need for large volumes of training data and high computational demands, especially when dealing with date-dependent features that are sparse and high-dimensional.
Innovation Solution
A computer-implemented method is introduced to generate embedding arrays representing date-related information, using a neural network that processes input data arrays with a lower number of dimensions than traditional feature arrays, thereby reducing computational demands and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manually-selected date-dependent features are used to augment time input in forecasting models, then the model can capture date-related patterns more effectively, but the dimensionality and sparsity of feature arrays increase, leading to higher computational resource demands
Solution Approach 1:
The patent transforms the high-dimensional sparse feature array into a lower-dimensional dense embedding array by introducing a new dimensional representation. The neural network learns to map date-dependent features from the original high-dimensional space into a compressed low-dimensional embedding space that preserves the essential temporal patterns while reducing computational complexity.
Solution Approach 2:
The patent changes the parameters of the feature representation by transforming discrete categorical features (day of week, month, holiday indicators) into continuous embedding vectors. This parameter transformation allows the model to capture date-related patterns more efficiently while reducing the overall dimensionality and sparsity of the input data.
2Loss of information
If high-dimensional sparse feature arrays are used as input to forecasting models, then comprehensive date information is provided, but the volume of training data needed increases and training time becomes prohibitive
Solution Approach 1:
The patent extracts only the essential date-related information needed for forecasting by projecting high-dimensional sparse feature arrays into lower-dimensional embedding arrays. The neural network learns to identify and extract the most salient temporal patterns from the comprehensive date features, discarding redundant information while preserving predictive power.
Solution Approach 2:
The patent performs preliminary dimensionality reduction and feature transformation before the main forecasting task. By pre-processing date-dependent features into compact embeddings, the model reduces the computational burden of subsequent training and inference operations, making the overall process more efficient without losing critical temporal information.
3Adaptability or versatility
If traditional feature arrays with many binary features are used, then detailed date attributes are represented, but the arrays become sparse with many zero entries, challenging optimization
Solution Approach 1:
The patent transforms binary categorical parameters into continuous embedding parameters. Instead of using sparse binary indicators for date attributes, the neural network learns continuous vector representations that capture the same information in a denser format, making the optimization landscape smoother and more tractable.
Solution Approach 2:
The patent creates composite embedding representations that combine multiple date-dependent features (day of week, month, holiday status) into unified vector representations. These composite embeddings integrate information from multiple feature sources while maintaining a compact dense structure that is more amenable to optimization than separate sparse binary features.
Data Source
AI summary
A computer-implemented method includes, for each of a set of training dates: receiving, for each of a sequence of dates including the training date, an input data array representing values of a predetermined set of date-dependent features; receiving a target output corresponding to an evaluation of a predetermined metric at the training date; and performing an update routine including processing the input data array for each date using first layers of a neural network, processing a resulting intermediate data array using second layers of the neural network to generate a network output, and updating values of parameters of the neural network in in a direction of a negative gradient of an error between the target output and the network output. The data processing system is then arranged to generate an embedding array by processing an input data array for each of a given sequence of dates using the first layers of the neural network.