Graph-Based Recommendations Service Using Compact Primary Memory Traversal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Collaborative filtering recommendation engines face challenges in making accurate recommendations with limited data and require significant computation, especially when dealing with a large number of users and items, leading to inefficiencies in operational complexity and user wait times.
Innovation Solution
A graph-based recommendations system that traverses a heterogeneous recommendations graph in real-time, using a compact representation stored in primary memory and employing a fast graph-traversing algorithm to reduce computational load and storage requirements, allowing for flexible adjustments to changing data contexts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If collaborative filtering recommendation engines pre-compute recommendations offline, then computational load is reduced, but user wait times increase and recommendation accuracy decreases with limited data
Solution Approach 1:
The system pre-computes and stores a compact representation of the recommendations graph in primary memory before requests arrive. This preliminary action enables fast real-time traversal without requiring offline batch pre-computation for each user, thus reducing user wait time while maintaining scalability.
Solution Approach 2:
The system dynamically traverses the pre-computed graph structure at request time based on specific user contexts and item characteristics, rather than using static pre-computed recommendations. This dynamic approach allows the system to adapt to changing data contexts and provide accurate recommendations even with limited user data.
2Reliability
If collaborative filtering engines collect and analyze large amounts of user behavior data, then recommendation accuracy improves, but device complexity and computational requirements increase
Solution Approach 1:
The recommendations graph is segmented into distinct node types (items, users, contexts) and relationship types. This segmentation allows the system to process and traverse only relevant portions of the graph for each recommendation request, reducing computational complexity while maintaining accuracy through targeted data analysis.
Solution Approach 2:
The patent introduces an intermediary compact graph representation that mediates between the raw large-scale user behavior data and the recommendation algorithm. This intermediary structure pre-organizes the data relationships, enabling efficient queries without requiring the system to process the entire raw dataset for each recommendation, thus reducing device complexity while preserving recommendation accuracy.
3Speed
If recommendation systems store detailed graph data in primary memory, then access speed improves, but storage requirements increase
Solution Approach 1:
The system creates a compact copied representation of the recommendations graph that preserves essential structural relationships and metadata while eliminating redundant information. This copied structure stores only the necessary edges, nodes, and weights needed for traversal, enabling fast access in primary memory without requiring storage of the complete detailed graph data.
Solution Approach 2:
The patent transforms the graph data into a compact representation by changing parameters such as storing condensed edge lists, using efficient data structures for node representations, and optimizing metadata storage. These parameter changes reduce the quantity of data stored in primary memory while maintaining the speed necessary for real-time graph traversal and recommendation generation.
Data Source
AI summary
A recommendation engine may provide recommendations by obtaining a compact graph representation representing a recommendations graph comprising of nodes and weighted edges. Each node is associated with type metadata indicating that it represents a recommendable item or a non-recommendable item. Each weighted edge is associated with edge-weight metadata. The compact graph representation can be stored in primary memory. When servicing a request for an item recommendation, the recommendation engine selects an entry node based at least in part on context metadata associated with the request, and traverses only a highly-weighted portion of the compact graph representation that is proximate to an entry node to select paths leading respectively to potential recommendation nodes. Each path is scored based on the edge-weight metadata of all segments, and at least one recommendation node is selected based at least in part on the path scores.