Deep and Wide Model for Job Recommendation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing job recommendation systems face challenges in accurately matching job postings with member profiles without relying on feature engineering, often resulting in over-generalization or high online scoring latency due to sparse features and poor semantic meaning capture.
Innovation Solution
A deep and wide machine learning model is employed, combining a deep neural network to generate dense features with a boosting decision tree to produce sparse wide features, eliminating the need for feature engineering and enhancing job posting recommendation performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If deep neural networks are used to generate dense embedding features, then generalization ability is improved and feature dimension is reduced, but over-generalization occurs and semantic meaning is lost
Solution Approach 1:
The patent merges deep neural networks (for generalization and dimensionality reduction) with gradient boosting decision trees (for semantic preservation and sparse feature generation). This combination allows the system to leverage the strengths of both models: the deep network's ability to generalize across data and the boosting tree's ability to capture semantic relationships and generate interpretable sparse features, thereby resolving the contradiction between generalization and semantic meaning preservation
Solution Approach 2:
The patent creates a composite machine learning model that integrates two different algorithmic approaches (deep learning and gradient boosting) into a unified recommendation system. This composite structure enables the system to simultaneously achieve the benefits of dense embedding features (generalization, low dimensionality) and sparse wide features (semantic meaning, interpretability) without relying on manual feature engineering
2Loss of information
If generalized linear models with nonlinearly transformed features are used, then over-generalization is reduced, but feature engineering is required and sparse features don't generalize well
Solution Approach 1:
The gradient boosting decision tree component automatically generates sparse wide features through its inherent feature transformation capabilities, eliminating the need for manual feature engineering. The model self-generates meaningful sparse features by learning optimal transformations and interactions from the data, while maintaining semantic meaning and generalization ability
Solution Approach 2:
The patent changes the approach to feature generation by using gradient boosting decision trees to automatically create sparse features from raw data, rather than relying on manual feature engineering or simple nonlinear transformations. This parameter change in the feature generation process enables the system to produce generalizable sparse features without human intervention
3Loss of information
If many sparse features are used in linear models, then semantic coverage is improved, but online scoring latency increases and fitting becomes difficult
Solution Approach 1:
The patent segments the feature representation into two distinct components: dense embedding features (generated by deep neural networks) for efficient generalization and low-dimensional representation, and sparse wide features (generated by gradient boosting decision trees) for semantic coverage. This segmentation allows the system to process features more efficiently by handling different types of features through appropriate model components, reducing online scoring latency while maintaining semantic coverage
Data Source
AI summary
In an example, features in a boosting decision tree model are initialized to zero, the boosting decision tree model located in a GLMM and connected to a deep neural network collaborative filtering model via a prediction layer. While the features in the boosting decision tree model remain zero, the deep neural network collaborative filtering model is trained. One or more trees in the boosting decision tree model are boosted using logits produced by the training of the deep neural network collaborative filtering model as a margin. The prediction layer is trained using features from the deep neural network collaborative filtering model and features from the boosting decision tree model. It is then determined whether a set of convergence criteria is met. If not, then the deep neural network collaborative filtering model is retrained using the features and the process is repeated until the set of convergence criteria is met.