Wide and Deep Machine Learning Model Architecture for Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning models either memorize feature interactions or generalize unseen combinations poorly, lacking a balanced approach to predict outputs effectively from input features.
Innovation Solution
Implementing a combined wide and deep machine learning model that includes a deep neural network for generalization and a wide model for memorization, with a combining layer to process intermediate outputs from both models, trained jointly using mini-batch stochastic optimization and adaptive learning rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a deep machine learning model is used, then generalization capability is improved, but memorization capability deteriorates
Solution Approach 1:
The patent combines a deep machine learning model and a wide machine learning model into a unified architecture. The deep model (with multiple hidden layers) provides generalization capability while the wide model (with cross-feature terms) provides memorization capability. Both models share the same input features and are trained jointly on the same dataset, with their outputs combined through a combining layer to produce the final prediction.
2Measurement precision
If a wide machine learning model is used, then memorization capability is improved, but generalization capability deteriorates
Solution Approach 1:
The patent merges the wide model's strength in memorization with the deep model's strength in generalization. The wide model processes input features including cross-product terms to capture feature interactions and memorize patterns, while the deep model simultaneously processes the same features through multiple non-linear transformations to generalize to unseen data. The combination allows both capabilities to coexist and complement each other.
3Reliability
If a combined wide and deep model is used, then prediction performance is improved, but model complexity increases
Solution Approach 1:
The patent segments the model into distinct functional components: a wide model segment for memorization, a deep model segment for generalization, and a combining layer segment for integration. Each segment has a specific role and can be independently understood and trained. The segmentation allows the complex functionality to be organized in a manageable way while maintaining the benefits of both model types.
Solution Approach 2:
Both the wide model and deep model share the same input feature set and are trained jointly on the same data, making them universal in their data processing approach. The combining layer universally integrates outputs from both models regardless of the specific application domain. This multi-functionality allows the combined architecture to handle diverse prediction tasks while maintaining a unified structure.
Data Source
AI summary
A system includes one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the computers to implement a combined machine learning model for processing an input including multiple features to generate a predicted output for the machine learning input. The combined model includes: a deep machine learning model configured to process the features to generate a deep model output; a wide machine learning model configured to process the features to generate a wide model output; and a combining layer configured to process the deep model output generated by the deep machine learning model and the wide model output generated by the wide machine learning model to generate the predicted output, in which the deep model and the wide model have been trained jointly on training data to generate the deep model output and the wide model output.