Gaussian Mixture Normalization for Non-Normal Time Series Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning systems face challenges with non-normal data distributions, leading to underfitting or overfitting, and require multiple models for each domain, resulting in high computational load and inefficiencies, especially with serial RNN architectures.
Innovation Solution
The system employs a Gaussian Mixture Model to approximate non-normal data distributions, trains a single LSTM model across multiple domains, and uses a parallel neural network architecture to improve prediction accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional normalization methods are used on non-normal data distributions, then the normalization process is simple, but prediction accuracy deteriorates due to underfitting or overfitting
Solution Approach 1:
The patent segments the non-normal data distribution into multiple clusters, each with its own normal distribution characteristics. By dividing the data into K clusters and applying normalization specific to each cluster rather than using a single global normalization, the system accurately captures the underlying structure of non-normal distributions, thereby improving prediction accuracy while maintaining manageable complexity through localized processing.
Solution Approach 2:
The patent changes the normalization parameters dynamically based on the identified cluster for each data point. Instead of using fixed global mean and standard deviation, the system adapts the normalization parameters (mean and standard deviation) to match the local characteristics of the cluster to which each data point belongs, thereby resolving the contradiction between accuracy and complexity.
2Measurement precision
If multiple machine learning models are trained for each domain of input data, then domain-specific prediction accuracy is improved, but computational load and memory storage requirements increase
Solution Approach 1:
The patent creates a universal normalization process based on Gaussian Mixture Models that can handle multiple domains of input data simultaneously. By learning a shared representation of the data distribution across domains through the GMM framework, the system eliminates the need to train separate models for each domain, thereby reducing computational load and memory requirements while maintaining the ability to handle domain-specific characteristics through the cluster assignment mechanism.
3Productivity
If serial RNN architectures are used for training machine learning models, then model training is straightforward, but performance delays and inefficiencies occur
Solution Approach 1:
The patent segments the sequential processing of serial RNNs into parallel processing streams by introducing multiple processing units that can simultaneously handle different aspects of the input data. Each processing unit operates independently on its assigned data stream, eliminating the sequential bottleneck while maintaining the recurrent processing capabilities needed for time series analysis, thereby improving training efficiency with moderate architectural complexity.
Data Source
AI summary
The disclosure relates to systems and methods of generating a mixture model for approximating non-normal distributions of time series data. The mixture model may include clusters of normal distributions that together approximate a non-normal distribution. The mixture model may be used to normalize input data for machine learning models. For example, a machine learning model such as an autoencoder may be trained to make predictions on the normalized input data. The predictions may relate to the time series of data. In one example, the time series of data may be market data for a security. The market data my include one or more features that are normalized using the mixture model. The predictions may include a predicted rate at which a lender will charge to borrow a security for short selling, where such rate may depend on the market data for the security.