Temporal Ensemble Neural Networks for Recommendation Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training neural networks for recommendation systems is time-consuming and computationally expensive, especially when dealing with massive datasets, and traditional ensembles require significant hardware resources.
Innovation Solution
Implementing a temporal ensemble of neural networks trained at different time intervals using overlapping but progressively updated datasets, where each neural network is trained with a sliding window of item interaction data, allowing for fewer processors to be used while maintaining accuracy and reducing overfitting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional neural network ensembles are used to improve prediction accuracy, then recommendation accuracy is improved, but computational cost and hardware resources increase significantly
Solution Approach 1:
The patent implements a dynamic ensemble where neural networks are trained at different time intervals and dynamically selected based on current input data characteristics. This allows the system to adaptively use multiple models only when beneficial, rather than statically maintaining N models simultaneously, thus reducing computational overhead while preserving accuracy benefits
Solution Approach 2:
The system employs periodic retraining of neural networks at different time intervals (e.g., daily, weekly, monthly) to create an ensemble of models with different temporal perspectives. This periodic action allows accurate predictions through model diversity without requiring all models to run simultaneously, reducing hardware requirements compared to traditional ensembles
2Productivity
If more processors are allocated to train multiple neural networks in parallel, then training speed is improved, but hardware cost increases proportionally
Solution Approach 1:
The patent segments the training process into multiple time-based batches, where different neural networks are trained on different temporal segments of the dataset. This segmentation allows sequential or staggered training of multiple models using the same hardware resources, avoiding the need for N times more processors while still achieving ensemble diversity
Solution Approach 2:
The system implements a sliding window approach where older training data is discarded after being used to train a model, and computational resources are recovered and reused for training subsequent models on newer data segments. This allows efficient resource utilization across multiple model training iterations without proportional hardware increases
Data Source
AI summary
The present disclosure is directed to training, and providing recommendations via, a temporal ensemble of neural networks. The neural networks in the temporal ensemble can be trained at different times. For example, a neural network can be periodically trained using current item interaction data, for example once per day using purchase histories of users of an electronic commerce system. The item interaction data can be split into a more recent group and a less recent group, for example the last two weeks of data and the remainder of the last two years of data. The periodic training of neural networks, using updated data and the sliding windows created by the date split, results in a number of different models for predicting item interaction events. Using a collection of these neural networks together as a temporal ensemble can increase recommendation accuracy without requiring additional hardware for training.